Попробую.
Из 1 уравнения
x + y + x^2 + y^2 = 8
Мы знаем, что
(x + y)^2 = x^2 + y^2 + 2xy
x^2 + y^2 = (x + y)^2 - 2xy
Подставляем
(x + y) + (x + y)^2 - 2xy = 8 (1)
Из 2 уравнения
xy^2 + yx^2 = 6
xy(x + y) = 6
xy = 6/(x + y) (2)
Подставляем уравнение (2) в уравнение (1)
(x + y)^2 + (x + y) - 12/(x + y) = 8
Замена (x + y) = t
t^2 + t - 12/t - 8 = 0
t^3 + t^2 - 8t - 12 = 0
Преобразуем так
t^3 - 3t^2 + 4t^2 - 12t + 4t - 12 = 0
(t - 3)(t^2 + 4t + 4) = 0
(t - 3)(t + 2)^2 = 0
t1 = x + y = 3
y = 3 - x
xy = 6/(x + y) = 6/3 = 2
x(3 - x) = 2
3x - x^2 = 2
x^2 - 3x + 2 = 0
(x - 1)(x - 2) = 0
x1 = 1; y1 = 3 - 1 = 2
x2 = 2; y2 = 3 - 2 = 1
t2 = x + y = -2
y = -2 - x
xy = 6/(x + y) = 6/(-2) = -3
x(-2 - x) = -3
-2x - x^2 = -3
x^2 + 2x - 3 = 0
(x + 3)(x - 1) = 0
x3 = -3; y3 = -2 + 3 = 1
x4 = 1; y4 = -2 - 1 = -3
Ответ: (1; 2); (2; 1); (-3; 1); (1; -3)