You have to pull it from the inboard end, which compresses its length to increase its diameter.
If you can't get it started, lever the inboard end with a screwdriver. Loosen the screw in the brake lever and slide it to the grip. If there's something in between like a throttle, loosen that too. Slide everything to the grip, retighten the clamp-screw in the brake and then lever between the brake clamp and everything outboard of it to slide them outboard. When you've levered a gap, loosen the brake clamp screw and slide the clamp until it's touching the grip again, then retighten it so that your can prise the grip along with the screwdriver again. Once you've moved it about an inch, you can pull it off with your hand from the inboard end. If you try and pull it from the middle of the grip, it won't come off. It'll just grip tighter like the "Chinese Finger" trick.