I think one of the problems with meditation is that external stimulation is at a minimum. What stimulation there is soon becomes habituated and ceases to register. For the mind then imagination, fantasies and daydreams can have a more, and a richer, reality than concentration on breathing or mantra. And so it becomes a continual struggle constantly bringing the mind back to attention.
A vague idea is running through my mind. One of the drawbacks, this is very tentative, of the Christian tradition with its anthropomorphic focus is that objects of devotion are often mental constructs, fantasies. Mystics, and those who have a religious experience, glimpse something of the reality behind these constructs. Others who have done some theology may be aware of the gap between the credal formula, the object of worship, and the transcendent reality. But for most the religious life is something that involves the whole person and especially the emotions and, while these intellectual considerations may be acknowledged, it is the emotive life which really matters and which gives flavour to living. To base this emotive life on mental constructs and fantasies seems to me to be laying up trouble for oneself. I am sure that much of the accedie contemplatives suffered from in the past, and perhaps John of the Cross’ Dark Night of the Soul, are due to this weaning away from a fantasy life.
In this respect Buddhism has much in its favour. The problem though is that we all need an emotive life, preferably one that is rich and satisfying. Can one find it in the austerity of a contemplative life? Perhaps yes. Perhaps it frees from fantasies, both those of the mind alone and those we project onto others, so that we can relate to others with greater inner freedom and with less clinging and grasping to a wishful desire that they be what we want them to be. We can acknowledge the other in all his/her uniqueness.