integral(0 <= y <= 1, y<= x <= 1) (x^2 + 5y)dxdy
= integral(0 <= y <= 1) (x^3/3 + 5xy) (for y<= x <= 1) dy
= integral(0 <= y <= 1) (1/3 + 5y) - (y^3/3 + 5y^2) dy
= 1/3 + 5/2 - 1/12 - 5/3
= 13/12.

Так?