All perfect praise be to Allah, The Lord of The Worlds; and may His blessings and peace be upon our Prophet Mohammad and upon all his family and companions.
If the couple made the intention to offer Umrah for Tamattu` Hajj, then lifted the restrictions of Ihram after having finished their Umrah, they are permitted to have sexual intercourse within the period between finishing Umrah and assuming a new Ihram for Hajj.
However, sexual intercourse renders Hajj invalid if it takes place before having lifted the restrictions of Ihram for Hajj. Allah, The Almighty, Said (What means): " When you are secure, then whoever enjoys the Visitation, until the Pilgrimage; let his offering be such as is feasible" {Al-Baqarah/196}.
Based on the above question, it seems that the questioner had sexual intercourse with his wife after having finished Umrah, but before assuming a new Ihram for Hajj, and during this period a pilgrim intending to offer Tamattu` Hajj is permitted to have sexual intercourse with his wife.
Therefore, your Hajj is valid and you are absolved from the liability. And Allah knows best.