Want Two came out a year after Want One, and the main distinction I’d draw between the two albums is that this one takes a more feminine perspective while the first was more masculine.
That dichotomy is borne out in the cover art… Want One depicting Wainwright dressed as a knight in a suit of armor while Want Two depicts him as a lady in waiting. In an embarrassing show of homophobia, the record company released Want Two with a giant sticker of critics quotes placed directly over Wainwright in drag. As if anybody buying his album would be scared off by that image?
Wainwright has always split his albums between the more arty stuff and catchy pop songs, and ‘The One You Love’ is the latter. It’s got an infectious, hypnotic beat, a great chorus and it moves like a mother. I could easily see this as a Top 40 hit, perhaps in an alternate universe where singer-songwriter types dominate the charts and record studio creations struggle to sell records.
This is a relatively straightforward breakup song, sung from the perspective of the guilty party: “No words ever could sell you on me after all that I have done.” I’m not sure exactly what the title phrase signifies. The “only” in “I’m only the one you love” seems sarcastic… is the person you love really something that can be limited in that way? I suppose that’s the point, though I’m not exactly sure what he’s getting at. Thoughts?
Why can’t I sleep with my eyes open?
The mind has so many memories
Can you remember what it looks like when I cry?
I’m trying, trying to tell you
All that I can in a sweet and velvet tongue
But no words ever could sell you
Sell you on me after all that I have done
I’m only the one you love
Am I only the one you love?
Lady Gloom and her hornets circling round
Is now before us, the screaming’s done without moving
One little move and for sure you will be stung
I’m singing “Oh, Jerusalem oh, Jerusalem
See what he’s picked up in the park”
Let’s fuck this awful art party
Want you to make love to me and only to me in the dark
I’m only the one you love
Am I only the one you love?
We’ve traded in our snap shots
We’re going through the motions
Into the view, I’m leaving you
Down Conduit Avenue into the early morning
Into the early morning
The one I love
Are you only the one I love?
Meet Me In Montauk 
I’m not sure why you’re convinced it’s sung from the perspective of the guilty party. It seems that it may be a mutual decision, as they trade in their snapshots and go through the motions (rather than one partner slinging said photos in the other’s face, for instance). Also, the refrain/title is approached from both directions – “Am I only the one you love?” and “Are you only the one I love?”
I take it as a person coming to terms with the end of what is perhaps a rather long relationship, trying to sort through the photos and the memories, and wondering what’s left at the end of it all. “Only” has such poignancy, because the line without it “You are the one I love” (or are you?)/ “I am the one you love” (or am I?) is tremendously powerful. Both as a statement and as a question. Adding the “only” is almost ironic. Is he dismissing the notion of love being something extraordinary or the notion that it is possible to love only one?
I’m not sure if they are breaking up or not, or whether it is mutual or not. I kinda think that he is someone who has and will inevitably hurt his significant other, and on the surface he acts callous and indifferent, generally tending to rationailze his actions, yet realizing deep down and in quieter retrospective moments that he is really being hurtful to someone who cares for him. So, he is being both sarcastic and self-revelatory in asking why she is so bothered that he is treating her like garbage since he’s “only” the one she loves. He knows he should be a better person to her, but can’t. He wants to be a better person for her, but can’t.
I like both of those interpretations better than mine. š