All perfect praise be to Allah the Lord of the Worlds. May His peace and blessings be upon our Prophet Mohammad and upon all his family and companions.
In principle, Islam calls for giving everyone their rights and performing duties of this life and the next. This is attested to in the following Hadith: Abu Juhaifah (May Allah be pleased with him) reported:
The Prophet (PBUH) made a bond of brotherhood between Salman and Abud-Darda'. Salman paid a visit to Abud-Darda' and found Umm Darda' (his wife) dressed in shabby clothes and asked her why she was in that state. She replied: "Your brother Abud-Darda' is not interested in (the luxuries of) this world. In the meantime Abud-Darda' came in and prepared a meal for Salman. Salman requested Abud-Darda' to eat (with him) but Abud-Darda' said: "I am fasting." Salman said: "I am not going to eat unless you eat." So, Abud-Darda' ate (with Salman). When it was night and (a part of the night passed), Abud-Darda' got up (to offer the night prayer) but Salman asked him to sleep and Abud-Darda' slept. After some time Abud-Darda' again got up but Salman asked him to sleep. When it was the last hours of the night, Salman asked him to get up and both of them offered (Tahajjud) prayer. Then Salman told Abud-Darda': "You owe a duty to your Rubb, you owe a duty to your body; you owe a duty to your family; so you should give to every one his due. Abud-Darda' came to the Prophet (PBUH) and reported the whole story. Prophet (PBUH) said, "Salman is right". [Al- Bukhari].
Therefore, a Muslim has to perform his duties towards wife and children in terms of caring and showing them love and kindness. Allah the Almighty Says {What means}: "on the contrary live with them on a footing of kindness and equity" {An-Nisa`, 19}.
Perhaps, the reward of kindness towards wife is greater than that of some voluntary acts of worship. Narrated 'Aishah: that the Messenger of Allah (PBUH) said: "The best of you is the best to his wives, and I am the best of you to my wives…" {Related by Tirmithi}.
Among the rights of the wife upon her husband is spending time with her, in addition to showing her love, kindness, compassion and affection. This relieves her from boredom, comforts her heart and motivates her to lead an active, happy life. Narrated Aisha, Ummul Mu'minin: while she was on a journey along with the Messenger of Allah (PBUH): I had a race with him (the Prophet) and I outstripped him on my feet. When I became fleshy, (again) I had a race with him (the Prophet) and he outstripped me. He said: This is for that outstripping." {Related by Abu Dawoud}.
However, we advise you (Asker) to fear Allah and put up with your husband for, as it seems from the above question, he is a pious man, and it is hard, nowadays, to find a man of such quality. Rather, maybe he just needs to be reminded of his duties towards you and his children. And Allah The Almighty Knows Best.