Miller was the son of Johann Muller, an engraver from Nuremberg, who settled in England in 1744. Of his twenty-seven children, two sons followed in their father’s career. It was probably through Banks acquaintance with his father that John Frederick came into Banks employment. He was given the task of making drawings of artefacts collected by Banks and others on the Endeavour’s voyage, and was to have accompanied Banks as a natural history draughtsman on Cook’s second voyage to the Pacific, on the Resolution, together with his brother James.