With all the attention rightly focused on the demise of the Minnesota Orchestra and the New York City Opera, it would be easy to believe that the musical arts are in dire straits in America. In some ways they are. But from the funding rubble of the financial crisis new groups and new facilities are sprouting too.

The huge Armory on Park Avenue, fairly low down on the Upper East Side, has been undergoing renovation since 2006. The New York Philharmonic played in its gigantic drill hall earlier this year, and slowly the rest of the building is coming to life. Among the first of the small rooms dotted around the building to be finished, the exquisite Board of Officers Room (completed in 1880 by influential Gilded Age architects, the Herter Brothers) has been painstakingly restored by Herzog & de Meuron. This small, plush room seats a couple of hundred at best in its deep leather chairs, and the restorers have done an enviable job not only in refurbishing the details, but in creating a sense of atmosphere. Deep red woods panel the royal green walls, brought back to life by intensive work with tens of thousands of tiny cotton buds. Chainmail of copper dangles over the windows, silently intimidating. Portraits of officers hang behind the baby Steinway, which in turn sits beneath an intimidatingly sharp chandelier composed of weaponry. In a town that lacks a great chamber hall such as London’s Wigmore Hall, this might well be it, despite its exclusive feel. In their steady work to turn swords into ploughshares, Park Avenue Armory and Herzog & de Meuron have made a start so winning that one can almost be certain, already, that the Armory will be a vital addition to the New York scene.

Especially so, if the artistic management can maintain the standards set by their first chunk of programming. Later in October Anton Batagov plays Triadic Memories, Morton Feldman’s vast work of piano and silence. Before that, burgeoning violinist Vilde Frang essays an all-Mozart programme of sonatas. To open this salon, though, Park Avenue Armory managed to bring baritone Christian Gerhaher and his pianist Gerold Huber across the Atlantic for an exceptionally rare appearance.

In a world of great baritones, Gerhaher is the baritone’s baritone. A Lieder singer above all, he is most often noted for his evenness of tone and his unfussy, almost confessional manner. Certainly that was on show here, whether in the two long-lined Myrthen to texts by Friedrich Rückert (“Aus den östlichen Rosen” and “Zum Schluss”), or in the utter stillness of the closing song of the twelve Kerner poems, “Alte Laute”. What surprised most in this all-Schumann programme was the wealth of power Gerhaher held in reserve, unleashing an unexpected ferocity for most violent moments. Take the end of the protagonist’s meeting with Lorelei in “Waldesgespräch” from the Liederkreis, or the scary forcefulness as man shot lion in a stunningly narrated “Die Löwenbraut”.

Even so, Gerhaher refuses to over-dramatise, maintaining such total control that individual words tell as they must. In “Wehmut” (here the emotional core of the Liederkreis) he somehow managed to crush the hope of the first stanza (“Da wird das Herz mir frei”, lightening on the “frei”) with an utterly hollowed out final line (“Im Lied das tiefe Leid”), sung as if sadness and song were one and the same. So they seem with Gerhaher, sometimes: his voice seems to have been transported here from another generation, but is often saturated with a profound sorrow, as if such things really ought not to be like this any more. In the Board of Officers Room – a throwback to the past in its salon feel but rigidly adhering to the lined chairs of 20th-century listening – that sense was only made more intense.

A word, too, for Gerhaher’s pianist, Gerold Huber. Often, in reviews of Liederabend, pianists are left aside, but it would be folly to do that here. With only a baby grand at his disposal and an unforgiving acoustic to play into, Huber’s sensitivity to both Schumann’s piano writing and to its relation to the text was remarkable. In the first of the Venetian songs from the Myrthen, he – like Gerhaher – somehow combined a swaggering confidence and a restless nervousness as the gondolier approached the evening’s love. In both “Zum Schluss” and “Alte Laute”, his terraced chords embraced both fragility and finality. His communication with Gerhaher, developed over long years working together, is essentially telepathic, to the complete benefit of the composer.

Faultless, all of it.