Several active areas of research in novel energy storage technologies, including three-dimensional solid state batteries and passivation coatings for reactive battery electrode components, require conformal solid state electrolytes. We describe an atypical atomic layer deposition (ALD) process for a member of the lithium phosphorus oxynitride (LiPON) family, which is employed as a thin film lithium-conducting solid electrolyte. The reaction between lithium tert-butoxide (LiOtBu) and diethyl phosphoramidate (DEPA) produces conformal, ionically conductive thin films with a stoichiometry close to Li2PO2N between 250 and 300 °C. Unusually, the P/N ratio of the films is always 1, indicative of a particular polymorph of LiPON that closely resembles a polyphosphazene. Films grown at 300 °C have an ionic conductivity of (6.51 ± 0.36) × 10-7 S/cm at 35 °C and are functionally electrochemically stable in the window from 0 to 5.3 V versus Li/Li+. We demonstrate the viability of the ALD-grown electrolyte by integrating it into full solid state batteries, including thin film devices using LiCoO2 as the cathode and Si as the anode operating at up to 1 mA/cm2. The high quality of the ALD growth process allows pinhole-free deposition even on rough crystalline surfaces, and we demonstrate the successful fabrication and operation of thin film batteries with ultrathin (<100 nm) solid state electrolytes. Finally, we show an additional application of the moderate-temperature ALD process by demonstrating a flexible solid state battery fabricated on a polymer substrate.