You thought of it 20 years ago and didn't do it, and you're considering it again now.
Instead of telling yourself it is too late to do it now, just go for it, so that in 20 years more you will be happy you planted this tree.
(Even if they are just "family" in the broader sense and not their own)