Morphological changes, e.g. thickness of retinal or choroidal layers in Optical coherence tomography (OCT), is of great importance in clinic applications as they reveal some specific eye diseases and other systemic conditions. However, there are many challenges in the accurate segmentation of retinal and choroidal layers, such as low contrast between different tissue layers and variations between images acquired from multiple devices. There is a strong demand on accurate and robust segmentation models with high generalization ability to deal with images from different devices. This paper proposes a new unsupervised guided adversarial adaptation (GAA) network to segment both retinal layers and the choroid in OCT images. To our best knowledge, this is the first work to extract retinal and choroidal layers in a unified manner. It first introduces a dual encoder structure to ensure that the encoding path of the source domain image is independent of that of the target domain image. By integrating the dual encoder into an adversarial framework, the holistic GAA network significantly alleviates the performance degradation of the source domain image segmentation caused by parameter entanglement with the encoder of the target domain and also improves the segmentation performance of the target domain images. Experimental results show that the proposed network outperforms other state-of-the-art methods in retinal and choroidal layer segmentation.