The parametric instability of planetary gears having elastic continuum ring gears is analytically investigated based on a hybrid continuous-discrete model. Mesh stiffness variations of the sun-planet and ring-planet meshes caused by the changing number of teeth in contact are the source of parametric instability. The natural frequencies of the time invariant system are either distinct or degenerate with multiplicity two, which indicates three types of combination instabilities: distinct-distinct, distinct-degenerate, and degenerate-degenerate instabilities. By using the structured modal properties of planetary gears and the method of multiple scales, the instability boundaries are obtained as simple expressions in terms of mesh parameters. Instability existence rules for in-phase and sequentially phased planet meshes are also discovered. For in-phase planet meshes, instability existence depends only on the type of gear mesh deformation. For sequentially phased planet meshes, the number of teeth on the sun (or the ring) and the type of gear mesh deformation govern the instability existence. The instability boundaries are validated numerically.