Depression, Party of Two?

This thou perceiv’st, which makes thy love more strong,
To love that well which thou must leave ere long

Fall is always a time of death and decay, in shades of lovely reds and yellows. Trees lose there leaves, and woodland creature get all set for their annual naps. The speaker of "That time of year" feels like his relationship is going through the same dying cycle. He feels himself withering way, but he also feels his mistress holding on. This leads me to believe that maybe the speaker is ill. However, I would not say he is morbid because Fall  is one of four seasons which cycle year round. This knowledge leads me to believe that the relationship is having some sort of rough patch but it will be reborn eventually. The last line of the poem (as seen above) indicates that the speaker is well aware that the relationship is strong at this point, so either he has soom unfortunate news to break to his maiden or he's depressed.