It sound like someone's by-passed the normal on switch. Normally there's a thin red wire that comes out of the controller to the "on" switch. The return from the switch activates the controller. I'm guessing that these wires were soldered together, although it's not possible to say without knowing why it was done or what was done.
Some bikes have the main switch on the battery. Maybe that was by-passed.