High entropy alloys (HEA) exhibit an excellent combination of properties, providing motivation to novel alloy development with single or multiphase microstructures. High temperature application of HEAs  requires comprehensive understanding of phases tability and environmental degradation. The equiatomic CrMnFeCoNi based alloy, also known as Cantor alloy, is a widely studied single-phase alloy for its superior structural properties. However, recent studies have demonstrated the formation of secondary phase during the annealing in the intermediate temperature ranges. We investigated the structural and chemical changes in Cantor alloy during elevated temperature oxidation using in situ and ex situ approaches along with the influence of Al addition to the alloy composition. We utilized Cr, Mn, Fe, Co and Ni L edge X-ray absorption near edge structure (XANES) measurements to understand the chemical changes in the material.  The structural and phase changes were analyzed using transmission electron microscopy (TEM) and atom probe tomography (APT). The oxidation states of the elements can be clearly identified from the XANES spectra and correlated to the phase information obtained from TEM and APT.  Addition of Al changes the oxide chemistry, material vulnerability to oxidation and these details will also be discussed.