Shiny things

I have a project where calculating the irradience map for anything metallic takes forever and it's driving me nuts. I've ended up replacing the materials with matte ones, just while I calculate the pass, but the camera keeps coming onto objects that I forgot about (it's a long shot), and I need to go back to "fix" those.

It's REALLY strange because when I merge the objects that have these materials into a blank scene and put them in a semi-similar scenario to test them out (lots of diffuse light coming through a door in a closed space) there's no problem at all.

One characteristic of all these materials is that they use falloff maps for coloring an reflectance.

Any idea how I can get around this?

Ooh, or maybe it's because of all the lights against the walls of the room, which don't really contribute to much more than a splash of light behind them....

Ugh. If that's the problem then I'm sorta screwed because, except for the lights and camera, the scene is x-refed.