Librivox: Starman's Quest by Silverberg, Robert

Summary: Travelling at speeds close to that of light, spacemen lived at an accelerated pace. When one of the twin boys left the starship, he grew older while his twin in space barely aged. So the starship twin left the ship to find what happened to his brother who was aging away on earth.