The story of Kaduveli Siddhar told by Sri Ramana Maharshi clearly proves that a spiritual seeker should not be bothered by what people say.
Kaduveli Siddhar was a saint who had an affair with a temple dancer and even had a child by her. When he was ridiculed for falling from his high state, he publicly composed and sang a song which ended with the following words: 'If it is true that I sleep night and day quite aware of my Self, may this stone burst into two and become the wide expanse!'
Immediately, the stone he indicated burst with a loud noise, much to the astonishment of the crowd which had gathered to deride him. Bhagavan's comment, after telling this story, was, 'He proved himself an unswerving jnani. One should not be deceived by the external appearance of a jnani.
Is there then any criterion by which the onlooker, with his ignorant perspective, can decide whether or not a person is realised? Not definitively, says Bhagavan, but he adds that there is one clue, one positive sign which should always be looked for:
The jnani's mind is known only to the jnani. One must be a jnani oneself in order to understand another jnani. However, the peace of mind which permeates the saint's atmosphere is the only means by which the seeker understands the greatness of the saint.
His words or actions or appearance are no indication of his greatness, for they are ordinarily beyond the comprehension of common people.