It still turns on when not charging, but the battery symbol will flash and it will shut down after 30 seconds or so. I don't even have time to take a picture! But it works perfectly fine if it's on the charger.
I've left it on the charger over night and also tried to charge just the battery over night. The battery doesn't seem to hold any charge.
Read more: My Nikon 5700 will only work if it's on the charger?