Is it permissible to swear by (Allah "The Living")?
Yes, it is permissible because "The Living" is one of Allah`s Attributes.
Is washing both openings (Istinjaa`) a must before every ablution?
Washing the front and back openings (Istinjaa`) is a must in case they have an impurity, but if they do not, there is no need for that, and Istinjaa` is not a condition for the validity of ablution.
I took a housing loan from the bank for a 20-year period. Four years have passed, and I recently learned that this loan is usurious. I wish to perform Umrah. Am I permitted to perform it? If not, what should I do?
You are permitted to perform Umrah. Strive to repay the debt in hopes of reducing the amount of usurious interest. Allah Says (What means): "And there are others who have acknowledged their sins. They had mixed a righteous deed with another that was bad. Perhaps Allah will turn to them in forgiveness. Indeed, Allah is Forgiving and Merciful." [At-Tawbah/102]. And Allah Knows Best.
I wish to Pray at the Mosque, but I couldn't because I am a Woman. How can I get such a Reward?
All perfect praise be to Allah The Lord of The Worlds, and may His peace and blessings be upon our Prophet Muhammad and upon all of his family and companions Abdullah Ibn Masood (May Allah be Pleased with him) reported that the Prophet (PBUH) said: "Her prayer inside her home is better than its courtyard, and her prayer in her own bedroom is better than inside her home." [Abu Dawood]. Her staying at home, and not going to the mosque in obedience to Allah`s injunctions entails a great reward similar to that bestowed on a man for going to the mosque since the Mercy and Blessings of Allah Prevails. And Allah Knows Best.