liftoff should be quick, although while I am convinced the 4000 had a purpose built delay (I have not proven this to myself yet) the later decks do not, so if there is a little bit of stickyness anywhere in the system it will appear to drag the cart out of the groove.
I would look at the where the plunger from the damper meets the cueing arm lever. Watch as this is released, and you will see where the sluggishness comes from, either the damper side or most likely the small L (it has another bend but I do not know of a charactor to portray its shape but you should find it easier enough!) shaped lever that lifts the tone arm in place. If it is slightly stiff it will seem to follow the damper after a short delay or or slower. A drop of oil in the levers pivot point (tucked under the sensor arm on left) should sort it out if this is the problem! Otherwise you may have to remove as with other examples but only as a last resort!