When you remove the crossover are there actually 2 pc boards that are held together by screws and standoffs? Then a set of pins that connect between the 2 boards? I think those sometimes oxidize and cause problems like you are describing. So take them apart, lightly rub/scrape the pins to remove corrosion. Spray with contact cleaner. Possibly slightly bend them so they will fit tighter when you put it all back together. Spray the the male AND female parts. Connect and disconnect them a couple of times to aid in the cleaning. The last time I messed with something similar I put some auto battery anti-corrosion spray on it before I put it back together.
If the speakers are accessible you can also send signal to them with the crossover in your hand and give some components the tap and wiggle test. It shouldn't take much pressure at all to cause a component with a bad connection to cut out. Of course you have to be careful to make sure the movement on one component isn't actually causing the one beside to move enough to go intermittent and it's really the problem one. Once you get the area where a broken/cold solder figured out you can generally tap lighter and lighter until you for sure find the one component that's really the problem since it should take the least tapping of all to show trouble. Then reflow it. If you're not sure, reflow everything in that area on the board.
Those have protection bulbs as well don't they? Make sure those are OK and the filament is intact. You might reseat them and spray the base and socket of them with contact cleaner as well.
The crimps on the connector where they crimp to the wires is always possible for the culprit as is the connector just being lose where it connects to the driver. But I suspect something more inline with the above will actually be the answer.