Threads on this topic come up every now and again, and they always lead to much amusement.
It's all about context and traditional usage. Until the 19th century, "doctor" traditionally referred to an academic or ecclesiastical doctorate (with some interesting variations
). Around the latter half of the 19th century, physicians appropriated the term, because they had begun to be professionally trained in schools which granted the MD. Fast forward to the present, and at least in the United States, "doctor" in ordinary usage refers to physicians.
While it's technically correct to call a Ph.D. a doctor, I find it jarring when people insist in mixed company that others call them that.