Два раза интегрируем по частям:
S ln^2(x)dx = ln^2(x)*x - Sx d ln^2(x) = ln^2(x)*x - S x*2*ln(x)*(1/x)dx =
= ln^2(x)*x - 2S ln(x) dx = ln^2(x)*x - 2ln(x)*x + 2Sx d ln(x) =
= ln^2(x)*x - 2ln(x)*x + 2S 1 dx = ln^2(x)*x - 2 ln(x)*x +2x + c =
= x(ln^2(x) - 2ln(x) + 2) + c