Pin and spring in crankcasefound when doing oil change

[Melter]

Hello,
I was doing an oil change this morning and when I removed the plug out came a dowel pin and spring. I think it is the crankshaft balancer pin. I am in the process of doing a complete overhaul and cosmetic redo. The engine ran fine before I started the bike. What do you think I should do now. I would not really like to take the whole engine apart to find that this has been running like this for some time. Opinions? Any advise would be greatly appreciated.