Maybe because you always need to speak Parsletounge to get in.
And even if you are a ghost you can't get through because of heaps of enchantments or something.
they could have run across in their travels of the sewer pipes, just did not register that is was the Chamber of Secrets, just another room in the castle, also needing to speak parceltongue to get in and or the enchantments protecting by slitherine and LV, probably all helped keep it a secret.
Linear Mode
