Srimad-Bhagavatam: Canto 5 - Chapter 1 - Verse 3
तथा चोक्तम्—न कुर्यात्कर्हिचित्सख्यं मनसि ह्यनवस्थिते ।यद्विश्रम्भाच्चिराच्चीर्णं चस्कन्द तप ऐश्वरम् ॥ ३ ॥
tathā coktam — na kuryāt karhicit sakhyaṁmanasi hy anavasthiteyad-viśrambhāc cirāc cīrṇaṁcaskanda tapa aiśvaram
All the learned scholars have given their opinion. The mind is by nature very restless, and one should not make friends with it. If we place full confidence in the mind, it may cheat us at any moment. Even Lord Śiva became agitated upon seeing the Mohinī form of Lord Kṛṣṇa, and Saubhari Muni also fell down from the mature stage of yogic perfection.
The first business of one trying to advance in spiritual life is to control the mind and senses. As Śrī Kṛṣṇa says in Bhagavad-gītā (15.7): Although the living entities are part and parcel of the Supreme Lord and are therefore in a transcendental position, they are still suffering in this material world and struggling for existence due to the mind and the senses. To get out of this false struggle for existence and become happy in the material world, one has to control the mind and senses and be detached from material conditions. One should never neglect austerities and penances; one should always perform them. Lord Ṛṣabhadeva personally showed us how to do this. In the Śrīmad-Bhāgavatam (9.19.17) it is specifically stated: A gṛhastha, vānaprastha, sannyāsī and brahmacārī should be very careful when associating with women. One is forbidden to sit down in a solitary place even with one’s mother, sister or daughter. In our Kṛṣṇa consciousness movement it has been very difficult to disassociate ourselves from women in our society, especially in Western countries. We are therefore sometimes criticized, but nonetheless we are trying to give everyone a chance to chant the Hare Kṛṣṇa mahā-mantra and thus advance spiritually. If we stick to the principle of chanting the Hare Kṛṣṇa mahā-mantra offenselessly, then, by the grace of Śrīla Haridāsa Ṭhākura, we may be saved from the allurement of women. However, if we are not very strict in chanting the Hare Kṛṣṇa mahā-mantra, we may at any time fall victim to women.