Cataracts are the most common blinding disease, and also impact the observation of the fundus. To boost the fundus examination of cataract patients, restoration algorithms have been proposed to address the degradation of fundus images caused by cataracts. However, it is impractical in clinics to collect paired or annotated fundus images for developing restoration models. In this paper, a restoration algorithm is designed for cataractous images without paired or annotated data. Domain generalization (DG) is applied to learn domain-invariant features (DIFs) from synthesized data, and the high-frequency components (HFCs) are extracted to conduct domain alignment. The proposed algorithm is used on unseen target data in the experiments. The effectiveness of the algorithm is demonstrated in the ablation study and compared with state-of-the-art methods. The code of this paper will be released at https://github.com/HeverLaw/Restoration-of-Cataract-Images-via-Domain-Generalization.