Is it permissible for a woman in her waiting period (Iddah) after her husband's death to leave her house and mix with non-Mahrams?

It is not permissible for a woman in her waiting period (Iddah) after the death of her husband to leave her house except for a necessity, such as for medical treatment, nor is she allowed to mix with non-Mahrams. A woman is not permitted to mix with anyone other than her husband or Mahrams, whether she is in her waiting period or not. However, there is no harm in accepting condolences from those who visit and responding to their inquiries about her well-being. And Allah Knows Best.

Is the father a Mahram (Non-marriageable) to his son`s mother-in-law?

The father is a non-Mahram (Marriageable) to his son`s mother-in-law, so it is impermissible for them to look at each other, or to have a seclusion (Khalwah).

Is it permissible for the big brother to force his sisters to wear Hijjab (Islamic wear) if their father didn`t do his part in this regard ?

All Perfect Praise is due to Allah, The Lord of The Worlds

The father`s attention should be drawn gently and politely, and the sisters should also be reminded with good words since Allah, The Almighty, Says (What means): "O ye who believe! Save yourselves and your families from a Fire whose fuel is Men and Stones." [Al-Tahreem/6]. And Allah Knows Best.

Am I permitted to perform more than a prayer within single ablution?

All perfect praise be to Allah, The Lord of The Worlds, and may His Peace and Blessings be upon our Prophet Muhammad and upon all of his family and companions.

You are permitted to pray as many as you could within one ablution and to recite Quran so long as your ablution isn't nullified. And Allah Knows Best.