Just fell over an interesting gear simulation maybe of interest to others.

[Nate]

My first thoughts on this are to calculate based on pitch diameters (equivalent to using un-toothed, smooth wheels) then look for a common factor for all the circumferences to decide on a tooth size. Iterate until the size/number of teeth is within machineable limits.

You can get approximate solutions that way.  For mathematically exact solutions you'd need to ensure that the tooth phase is rational, and that's not going to be possible with a numerical method like the one you describe.