Brake advice to cure my insanity

[itsagz250butitsmine]

Hello guys, I've been lurking a while and got a lot of helpful advice and info from all of you already that saved me crazy amounts of time and money. I want to say thank you to the community. Anyway, I decided to join to ask a question that I wasn't able to find an answer for. I got this bike that was neglected and fixed everything except the rear brake. The brake would drag and not spring back like it should so I disassembled the brake and cleaned everything inside and out, greased pin and all contact points of movable parts. After re-assembly it's was finally springing back like it should by itself with me helping it. However, it has a hard time pulling the brake cable back with it unless I give it a little bit of help. I already disconnected both sides of the brake cable and was able to drip a very thin oil down the whole thing until it came out of the other side and kept working the cable back and forth to make sure the oil got everywhere. Anyway, it still can't pull the cable back by itself. My question is what the heck do I do because I'm not sure if the cable is supposed to have a little friction, or its fine, or if the brake shoe springs are too weak, or its fine. Sorry for the novel I just tried to provide the most complete information as I know how valuable that is.

Thanks guys