Question
Who was Lord Hayagriva?
Answer
Yes, Lord Hayagriva is the avatar of Vishnu with a horse head. Haya means horse and griva means neck. So the deity with a horse head from his neck is known as Hayagriva. The incident of how He got that horse head is present in different scriptures with slight variations.
Once in the ancient time the gods decided to do a Yajna. But they couldn't find where was Vishnu. They searched for Him at different places, and then after meditating they got to know where He was present. When they got there they saw that Shri Vishnu was in meditation supporting His head on His bow. To wake Him up, Brahma created some ants (or that ant like creatures that eat away wood and things) and told them to eat away one part of the bow so that when it breaks the Lord's meditation will break by losing the support. However, when it was done, the Lord's head got cut up getting tangled with the string of the bow. Then to correct it a horse head was fixed there by the god Viswakarma. After that incident Vishnu did penance and got back His original head. [Skanda Purana - 3.2.14,15 / Devi Bhagavata - 1.5]
The scriptures also mention the reason why the Lord's head was lost. But the reasons given are different. While Skanda purana states it was due to a curse from Brahma, the Devi Bhagavata says it was due to a curse from Laxmi and also because a demon by the same name who steals the Vedas from Brahma eventually (SB - 8.24), has asked the boon to be killed by someone who had a horse head.
So overall, even the accounts differ slightly in the scriptures, Hayagriva is the horse headed incarnation as you have heard.
Giving Vedas to Brahma
When Madhu and Kaitabha stole the Vedas from Brahma, He become overtaken by grief and requested Lord Vishnu to rescue the Vedas. So the Lord as Hayagriva brings the Vedas back and gives it to Brahma:
The two Asuras, making an appointment with the Vedas in respect of the time when they would come back to take them up again, threw them down in the nether region, and ran towards the spot whence those sounds appeared to come. Meanwhile, O king, the Supreme Lord with the equine head, otherwise called Hari, who was himself in the nether region, took up all the Vedas. Returning to where Brahma was staying, he gave the Vedas unto him. Having restored the Vedas unto Brahma, the Supreme Lord once more returned to his own nature.
Having compassed the destruction of the two Asuras and restored the Vedas to Brahma, the Supreme Being dispelled the grief of Brahma. Aided then by Hari and assisted by the Vedas, Brahma created all the worlds with their mobile and immobile creatures. After this, Hari, granting unto the Grandsire intelligence of the foremost order relating to the Creation, disappeared there and then for going to the place he had come from. It was thus that Narayana, having assumed the form equipt with the horse-head, slew the two Danavas Madhu and Kaitabha (and disappeared from the sight of Brahma).
Reference
Mahabharat - 12.348
Note: “The question: Who was Lord Hayagriva?” is licensed by Stack Exchange Inc (https://hinduism.stackexchange.com/); user contributions licensed under CC BY-SA.