Cos x = t, dt = -sin x dx, sin^3 x dx = sin^2 x*sin x dx = -(1 - t^2) dt
Int (t^2 - 1)/(t - 3) dt = Int (t^2 - 6t + 9 + 6t - 9 - 1)/(t - 3) dt =
= Int [(t - 3)^2 + (6t - 10)] / (t - 3) dt = Int (t - 3) dt + Int (6t - 18 + 8)/(t - 3) dt =
= t^2/2 - 3t + 6 + Int 8/(t - 3) dt = t^2/2 - 3t + 6 + 8*ln|t - 3| + C
Подставь обратно t = cos x