It's also illegal to ride in any place accessible to the general public, such as bridleways, cycle paths, canal paths, etc. Only legal on private land with no right of public access, and with the permission of the landowner.
I'm not sure about the rules for BOATs (byways open to all traffic)...