How can I make this peach tree grow?

My fianc� and I are moving to a new home, and there's a fairly large peach tree in the back yard. It's about 10-12' tall and has small green peaches on it but not many leaves. The previous owners said the tree grows but the peaches never get any bigger or change color. What can I do to make this tree flourish? Keep in mind I'm really not plant-savvy, so try and keep it simple :)