You need to remove the rear door card first. Theres a couple of screws behind that need to be undone, then just pull the pillar off.
I think thats how you do it, I had mine off to clean them a while back, but I can't remember the exact way to remove them.