Butler was a sketcher and seaman, who served aboard the Prince of Wales in the First Fleet. His cousin Daniel Southwell was midshipman in the Sirius . When Southwell received a report from ‘one of our Gents, who came a good part of the voyage in the P.W.’ that Butler was ‘quite the servant and bang’d about (I think) by the mate’, Southwell told his fellow midshipman Matthew Alt to ask his uncle, the surveyor-general-to-be Augustus Alt , to watch out for young Butler’s interests.