Well, if you have osteophytes, I don't see how he/she would say that you don't have impingement! Any time there is anything compromising the rotator cuff tendons, whether it is a type III acromion, or osteophytes, or poor mechanics resulting in decreased subacromial space, and they cause pain with shoulder movement, in my book, that is impingement.
Here is some good info from the AAOS.
The problem with letting it go too long when you have osteophytes is that the tendons eventually get "worn down" by the bone rubbing on them, leading to further rotator cuff pathology and eventually tears. If there are no tears, the surgery to trim away the spurs is frequently done arthroscopically, and recovery is fairly easy.
Have you seen another surgeon for a second opinion?
You can
search for a PT on the APTA website, which is not always a guarantee of a quality therapist.
Here are some things to look for in a good physical therapist ...