Used to hear this all the time when I worked phones for an ISP.
Yes, change your password, it's never a bad idea to do so, but the odds are it's just a spammer forging a "FROM" address as mentioned.
Spammers send to enormous lists of addresses, a great many of which are not legit or active addresses. Each email sent to a bad address generates a bounce reply. If they use their own address, the replies will take them down. Instead, they use a constantly changing set of false "FROM" addresses, both to give their emails some hint of legitimacy, and to redirect the bounces.
The problem is that your inbox is going to fill up with garbage, and the only way to stop it is to ask some scumbag scammer spammer asshole very politely to use another address.
Either you or your mail admin could cook something up that disposes of them automatically, but that's more just a band aid.
Usually they move on to a new victim in short order, but I've heard of some folks who had ongoing problems that were not solved except by abandoning the affected address.