I’d check a couple of easy things before buying a new battery. Sometimes the charger shows the light but isn’t actually charging, so if you know someone with a multimeter or another charger, that’s worth testing. Also make sure the battery connections are clean and tight.
If the battery has been sitting flat for a long time, it might just need a reset or a proper charge from an e-bike shop. A local bike shop should be able to test whether it’s the charger or the battery so you don’t waste money replacing the wrong part.