What is the ruling on a woman shaking hands with the brothers of her husband`s father, and not putting on Islamic wear before them?

The aforesaid are strangers to the their brother`s son`s wife, therefore, it is forbidden for her to take off her Islamic wear before them, or to shake hands with them.

What should a praying person who doubts having offered one, or two prostrations in the third Rakah (unit of prayer) do?

Whoever has doubts about their prayer should consider what is less i.e. the above person should offer another Sajda (prostration) and complete his/her prayer, then offer Sujood As-Sahw (prostration of forgetfulness). And Allah Knows Best.

Is it permissible for us to sever ties of kinship if our blood-relatives` gatherings/meetings involve acts of sin?

Observing kinship ties is obligatory, and paying your blood-relatives a visit causing their sinful acts to be hindered , then you should do so. However, if their sinful acts continue while you are at their gathering and they didn't respond positively, then observing kinship ties via telephone and the like will suffice in this case. And Allah Knows 

Is wearing a cap permissible for boys, or girls?

All Perfect Praise is due to Allah, The Lord of The Worlds.

In principle, anything is permissible sol long as there is no Sharia evidence prohibiting it, thus wearing the cap by boys is permissible as long as it isn`t done in imitation of the disbelievers. Moreover, a Muslim woman should avoid all that which attracts the attention of men because whatever leads to the forbidden is forbidden. And Allah Knows Best.