All perfect praise be to Allah the Lord of the Worlds. May His peace and blessings be upon Prophet Mohammad and upon all his family and companions.
We (Iftaa` Department) extend Our thanks to you for being keen on pleasing your brothers, seeking what is lawful, avoiding what is unlawful and clearing your father from liability.
Muslim jurists (May Allah have mercy on them) are of the view that it is desirable for a child to return the gift, which his/her father had given him/her while not in need (Sick, indebted and the like) since that is closer to achieving justice, reconciling hearts, and clearing oneself from liability.
This is particularly true if your father (Asker) had loaned you that amount then told you that he didn`t want it back; in which case the rest of the heirs may not believe this even though you are telling the truth. Therefore, we recommend that you resort to peaceful means to arrive at a certain agreement with them to avoid dispute.
As regards incurring sin, if you are sure that your father had told you that the money is yours-as mentioned in your question-We pray that Allah doesn`t hold you or him to account for that. This is provided that he had forgiven you willingly and in his full capacity considerable in Sharia. And Allah The Almighty Knows Best.