A colleague of mine was struggling with numerous issues when using XREFs. The two main ones where having multiple Layer States with XREF prefixes that couldn’t be removed and the other was not wanting to see the XREF layers.

The first issue was frustrating as whilst there is an option to disable these is in the layer states manger dialogue, checking the box does not disable them in the Ribbon pulldown. Duh. So the solution ended up being, delete the layer states in the XREF.
The second issue was easy to solve as AutoCAD has this feature built-in and it works! It works with both the Layer Pulldown, Layer Palette and Classic Layer! Wow!

This is how to remove XREF layers from these palettes. You will need a filter (yup AutoCAD has layer filters!)

On the panel on the left, click XREF, this will now show the list of layers for the XREF. Click on the checkbox below that states Invert Filter and voila, you have a list of layers without XREF layers within.