It's not likely to be the motor nor the controller. First disconnect the brake connectors from the controller to see if gets better. After that, I'd short out the key-switch to see if that improves anything. I guess that you've checked that all battery connections are tight. Can you show a photo of your display on the handlebars?