Can ingesting Neosporin harm my baby?

So before you answer let me say "Yes, I'm an idiot" I've been having some serious chapped lip issues and I read online that I should try neosporin and so I started using it yesterday and now I read that It's not supposed to be ingested. I'm 6 months pregnant and worried about my little one. You have to understand my lips are seriously raw and I've been using it to protect and (from what I heard) heal them so I know I ingested a bit just like you chapstick when you use it. My hubby swears it's ok and that I should just not use it anymore... can anyone put my mind at ease? I go to the doc on Thursday so will check with her then... but thats 4 days away!!!