Cos(3п/7) = cos(п - (4п/7)) = - cos(4п/7);
cos(5п/7) = cos(п - (2п/7)) = - cos(2п/7);
Поэтому
cos(п/7)*cos(3п/7)*cos(5п/7) = cos(п/7)*cos(2п/7)*cos(4п/7) =
= ( sin(п/7)*cos(п/7)*cos(2п/7)*cos(4п/7) )/sin(п/7) =
= (1/2)*sin(2п/7)*cos(2п/7)*cos(4п/7)/sin(п/7) =
= (1/2)*(1/2)*sin(4п/7)*cos(4п/7)/sin(п/7) =
= (1/2)*(1/2)*(1/2)*sin(8п/7)/sin(п/7) =
= (1/8)*sin( п + (п/7) )/sin(п/7) = (1/8)*(- sin(п/7) )/sin(п/7) =
= -1/8.