It's not just curve, it's also a climb and a drop. Particularly in the flying junction on the NJ side of the tunnel.

Speed control on a train is drastically impaired when a train is not level. That's because the wheels have very little grip on the track. Remember that minimal friction allows it to move so much weight with minimal power also means it doesn't have much control.

That's why track anywhere near a curve or station tends to be setup to be as flat as possible. But occasionally compromises have to be made. Less curves can only be done if the turn radius is larger, meaning more track distance, moving stations, potentially more obstacles. Even if the train runs faster, the distance can wipe away most of the time improvement.