Yes, adding in an actual prim within/around the column should fix the issue you are having. It sounds like the physics shape for the column is likely just thin/small enough for the cat to poke through and hide.
“What greater gift than the love of a cat.” ― Charles Dickens
