Although lineage-specific genes have been identified in the mammary gland, little is known about the contribution of 3D genome organization to gene regulation in the epithelium. Here we describe the chromatin landscape of the three major epithelial subsets through integration of long- and short-range chromatin interactions, accessibility, histone modifications and gene expression. We find that lineage specific marker genes display unique epigenetic modifications, achieving lineage specification within the mammary gland. Basal genes display exquisite lineage-specificity via coupling of faithful promoter and distal enhancer activity. Luminal-specific genes show widespread promoter priming in basal cells. Cell-specificity in luminal progenitors is largely mediated through extensive chromatin interactions with super-enhancers in gene-body regions, in addition to interactions with polycomb silencer elements. Genes enriched in mature luminal cells strangely display increased bivalent chromatin across their promoters which are linked to distal active enhancers through low-level chromatin interactivity. Finally, histone modifications were largely homogenous between active and quiescent basal cells, with chromatin accessibility proving the most dynamic epigenetic factor. Interestingly, genes enriched in quiescent cells were highly enriched for polycomb across both basal populations. This work provides a comprehensive resource for understanding the role of higher-order chromatin interactions in cell-fate specification and differentiation in the adult mouse mammary gland.