If I come to your house and tell you to change your air-freshener cause I don't like how the current one smells, are you being
selfish if you choose to not to change it?

Depends.

Since you like to reason by analogy, let's extend the point. If your air freshener (or by extension, toxic waste, or excessively loud music, or noisy/vicious dogs, or procession of crack-heads visiting your home at all hours to buy drugs, etc.) is extreme enough to damage me or reduce the value of my property -- then I have not only the right but the neighborly duty to request that you change your behavior.

Freedom and rights of ownership extend only to the point where you are not causing harm to other people. It is pretty hard to meet that criterion when it comes to using tobacco.

tanstaafl.