a: \(x^2+y^2(x-y+1)-\left(x-1\right)\cdot y=22\)

=>\(x^2 + y^2(x - y + 1) - xy + y = 22\)

=>\(x^2-xy+y^2(x-y+1)+y=22\)

=>\(x(x-y+1)-x+y^2(x-y+1)+y=22\)

\(\iff(x-y+1)(x+y^2)-(x-y)=22\)

\(\iff(x-y+1)(x+y^2)-(x-y+1)+1=22\)

=>\((x-y+1)(x+y^2-1)=21\)

TH1: \(\begin{cases}x-y+1=1\\ x+y^2-1=21\end{cases}\)

=>\(\begin{cases}x=y\\ y+y^2-1=21\end{cases}\iff y^2+y-22=0\)

=>\(y^2+y+\frac14-\frac{89}{4}=0\)

=>\(\left(y+\frac12\right)^2=\frac{89}{4}\)

=>\(\left[\begin{array}{l}y+\frac12=\frac{\sqrt{89}}{2}\\ y+\frac12=-\frac{\sqrt{89}}{2}\end{array}\right.\Rightarrow\left[\begin{array}{l}y=\frac{\sqrt{89}-1}{2}\left(loại\right)\\ y=\frac{-\sqrt{89}-1}{2}\left(loại\right)\end{array}\right.\)

=>Loại

TH2: \(\begin{cases}x-y+1=21\\ x+y^2-1=1\end{cases}\iff\begin{cases}x=y+20\\ (y+20)+y^2-1=1\end{cases}\iff y^2+y+18=0\)

=>\(y^2+y+\frac14+\frac{71}{4}=0\)

=>\(\left(y+\frac12\right)^2+\frac{71}{4}=0\) (vô lý)

=>Loại

TH3: \(\begin{cases}x-y+1=3\\ x+y^2-1=7\end{cases}\iff\begin{cases}x=y+2\\ (y+2)+y^2-1=7\end{cases}\iff y^2+y-6=0\)

=>(y+3)(y-2)=0

=>y=-3 hoặc y=2

Nếu y=-3 thì x-y+1=3

=>x-y=2

=>x-(-3)=2

=>x+3=2

=>x=-1

Nếu y=2 thì x-y+1=3

=>x-2+1=3

=>x-1=3

=>x=4

TH4: \(\begin{cases}x-y+1=7\\ x+y^2-1=3\end{cases}\iff\begin{cases}x=y+6\\ (y+6)+y^2-1=3\end{cases}\iff y^2+y+2=0\)

=>y∈∅

=>Loại

\(y\left(x-1\right)=x^2+2\)

\(\Leftrightarrow x^2-xy+y+2=0\)

\(\Leftrightarrow x\left(x-1\right)-y\left(x-1\right)+\left(x-1\right)+3=0\)

\(\Leftrightarrow\left(x-1\right)\left(x-y+1\right)=-3\)

\(\Leftrightarrow\left[{}\begin{matrix}\left\{{}\begin{matrix}x-1=-1\\x-y+1=3\end{matrix}\right.\\\left\{{}\begin{matrix}x-1=3\\x-y+1=-1\end{matrix}\right.\\\left\{{}\begin{matrix}x-1=1\\x-y+1=-3\end{matrix}\right.\\\left\{{}\begin{matrix}x-1=-3\\x-y+1=1\end{matrix}\right.\end{matrix}\right.\)\(\Leftrightarrow\left[{}\begin{matrix}\left\{{}\begin{matrix}x=0\\y=-2\end{matrix}\right.\\\left\{{}\begin{matrix}x=4\\y=6\end{matrix}\right.\\\left\{{}\begin{matrix}x=2\\y=6\end{matrix}\right.\\\left\{{}\begin{matrix}x=-2\\y=-2\end{matrix}\right.\end{matrix}\right.\)

Vậy \(\left(x;y\right)\in\left\{\left(0;-2\right),\left(4;6\right),\left(2;6\right),\left(-2;-2\right)\right\}\)

Ta có \(y\left(x-1\right)=x^2+2\)

\(\Leftrightarrow y\left(x-1\right)-x^2=2\)

\(\Leftrightarrow y\left(x-1\right)-x^2+1=3\)

\(\Leftrightarrow y\left(x-1\right)-\left(x^2-1\right)=3\)

\(\Leftrightarrow y\left(x-1\right)-\left(x-1\right)\left(x+1\right)=3\)

\(\Leftrightarrow\left(x-1\right)\left(y-x-1\right)=3\)

Vì x,y nguyên nên ta có bảng

Vậy\(\left(x,y\right)=\left\{\left(4,6\right),\left(2,8\right),\left(0,2\right),\left(-2,4\right)\right\}\)thỏa mãn

\(x^2+x+13=y^2\\ \Leftrightarrow x^2-y^2+x+13=0\\ \Leftrightarrow4x^2-4y^2+4x+52=0\\ \Leftrightarrow\left(2x+1\right)^2-4y^2=51\\ \Leftrightarrow\left(2x+1-2y\right)\left(2x+1+2y\right)=51=51\cdot1=17\cdot3\left(x,y>0\right)\)

Tới đây giải ra các trường hợp thui

Áp dụng bđt AM-GM ta có \(\left(x^2+1\right)\left(x^2+y^2\right)\ge2x.2xy=4x^2y..\)
\(\Rightarrow VT\ge VP\)
Dấu = xảy ra khi \(\hept{\begin{cases}x^2=1\\x^2=y^2\end{cases}\Rightarrow}\left(x,y\right)\in\left\{\left(1;1\right);\left(1;-1\right);\left(-1;1\right);\left(-1;-1\right)\right\}\)

Đặt \(S=x+2y\Rightarrow x=S-2y\)

Xét 2 trường hợp :

TH1: \(x^2+y^2>1\)từ giả thiết \(\Rightarrow x^2+y^2\le x+y\Leftrightarrow\left(S-2y\right)^2+y^2\le S-y\Rightarrow5y^2-\left(4S-1\right)y+S^2-S\le0\left(1\right)\)

Coi (1) là bất pt bậc 2 đối với ẩn y 

\(\Rightarrow\Delta=\left(4S-1\right)^2-20\left(S^2-S\right)\ge0\Rightarrow4S^2-12S-1\le0\Rightarrow S\le\frac{3+\sqrt{10}}{2}\)

Đẳng thức xảy ra khi \(x=\frac{5+\sqrt{10}}{2}\) thỏa mãn \(x^2+y^2>1\)

Vậy \(S_{m\text{ax}}=\frac{3+\sqrt{10}}{2}\)

TH2: Nếu \(x^2+y^2< 1\Rightarrow x+y\le x^2+y^2\)\(\Rightarrow S=x+2y\le x^2+y^2+y< 1+1=2\Rightarrow S< \frac{3+\sqrt{10}}{2}\)

Vậy S lớn nhất là \(\frac{3+\sqrt{10}}{2}\)khi \(x=\frac{5+2\sqrt{10}}{10};y=\frac{5+2\sqrt{10}}{10}\)

Từ phương trình \(y\left(x-1\right)=x^2+2\Rightarrow x^2+2\vdots x-1\to x^2-1+3\vdots x-1\to3\vdots x-1\to x-1=\pm1,\pm3.\)

Do vậy mà \(x=2,0,4,-2\).  Tương ứng ta có \(y=6,-2,6,-2\)

Vậy các nghiệm nguyên của phương trình \(\left(x,y\right)=\left(2,6\right),\left(0,-2\right),\left(4,6\right),\left(-2,-2\right).\)