\(3x+y+6z\le5xyz+5xz^2+x^2y+x^2z=x\left(y+z\right)\left(5z+x\right)\)

\(\Rightarrow3x+y+6z\le\frac{1}{2}.2x\left(y+z\right)\left(5z+x\right)\le\frac{1}{54}\left(3x+y+6z\right)^3\)

\(\Rightarrow\left(3x+y+6z\right)^2\ge54\)

\(\Rightarrow3x+y+6z\ge3\sqrt{6}\)

\(\Rightarrow P\ge6\sqrt{6}\)

\(x(3 - xy - xz) + y + 6z \le 5xz(y + z) \quad (1)\)

=>\(3x - x^2 y - x^2 z + y + 6z \le 5xyz + 5xz^2\)

=>\(3x+y+6z\le x^2y+x^2z+5xyz+5xz^2\)

=>\(3x+y+6z\le x^2(y+z)+5xz(y+z)\)

=>\(3x+y+6z\le x(y+z)(x+5z)\quad(2)\)

P=3x+y+6z

=>P<=x(y+z)(x+5z)

y+6z=P-3x

=>y+z=(P-3x)-5z

(2)=>\(P \le x(P - 3x - 5z)(x + 5z)\)

Theo AM-GM, ta có: \((P - 3x - 5z)(x + 5z) \le \left[ \frac{(P - 3x - 5z) + (x + 5z)}{2} \right]^2 = \left( \frac{P - 2x}{2} \right)^2\)

=>\(P \le x \cdot \left( \frac{P - 2x}{2} \right)^2 = \frac{x(P - 2x)^2}{4}\)

=>\(4P\le x(P^2-4Px+4x^2)\)

=>\(4P\le P^2x-4Px^2+4x^3\)

=>\(P^2x-4Px^2-4P+4x^3\ge0\)

=>\(P^2 - 4P \left(x + \frac{1}{x}\right) + 4x^2 \ge 0 \quad (3)\)

Theo Cosi, ta có: \(x + \frac{1}{x} \ge 2\sqrt{x \cdot \frac{1}{x}} = 2\)

=>\(-4P\left(x + \frac{1}{x}\right) \le -4P \cdot 2 = -8P\)

(3)=>\(P^2 - 8P + 4x^2 \ge P^2 - 4P \left(x + \frac{1}{x}\right) + 4x^2 \ge 0\)

Chọn điểm rơi x=1, ta sẽ có phương trình: \(P^2 - 8P + 4 \ge 0\)

=>\(P^2-8P+16\ge12\)

=>\(\left(P-4\right)^2\ge12\)

=>\(P-4\ge2\sqrt3\)

=>\(P\ge4+2\sqrt3\)

Dấu '=' xảy ra khi \(x=\frac{1}{x};P-3x-5z=x+5z\) và y=P-3x-6z

=>x=1; P-4x=10z và y=P-3x-6z

=>x=1; \(10z=(4+2\sqrt{3})-4=2\sqrt{3}\) và y=P-3x-6z

=>x=1; \(z=\frac{\sqrt{3}}{5}\) và y=P-3x-6z

=>\(x=1;y=1+\frac{4\sqrt{3}}{5};z=\frac{\sqrt{3}}{5}\)

Áp dụng BĐT Cô - si cho 3 bộ số không âm

\(\Rightarrow\frac{z\left(xy+1\right)^2}{y^2\left(yz+1\right)}+\frac{x\left(yz+1\right)^2}{z^2\left(xz+1\right)}+\frac{y\left(xz+1\right)^2}{x^2\left(xy+1\right)}\ge3\sqrt[3]{\frac{xyz\left(xy+1\right)^2\left(yz+1\right)^2\left(xz+1\right)^2}{x^2y^2z^2\left(yz+1\right)\left(xz+1\right)\left(xy+1\right)}}=3\sqrt[3]{\frac{\left(xy+1\right)\left(yz+1\right)\left(xz+1\right)}{xyz}}\)

Xét \(3\sqrt[3]{\frac{\left(xy+1\right)\left(yz+1\right)\left(xz+1\right)}{xyz}}\)

\(=3\sqrt[3]{\left(\frac{xy+1}{x}\right)\left(\frac{yz+1}{y}\right)\left(\frac{xz+1}{z}\right)}\)

\(=3\sqrt[3]{\left(y+\frac{1}{x}\right)\left(z+\frac{1}{y}\right)\left(x+\frac{1}{z}\right)}\)

Áp dụng BĐT Cô - si

\(\Rightarrow\left\{\begin{matrix}y+\frac{1}{x}\ge2\sqrt{\frac{y}{x}}\\z+\frac{1}{y}\ge2\sqrt{\frac{z}{y}}\\x+\frac{1}{z}\ge2\sqrt{\frac{x}{z}}\end{matrix}\right.\)

\(\Rightarrow\left(y+\frac{1}{x}\right)\left(z+\frac{1}{y}\right)\left(x+\frac{1}{z}\right)\ge8\)

\(\Rightarrow3\sqrt[3]{\left(y+\frac{1}{x}\right)\left(z+\frac{1}{y}\right)\left(x+\frac{1}{z}\right)}\ge3\sqrt[3]{8}\)

\(\Rightarrow3\sqrt[3]{\left(y+\frac{1}{x}\right)\left(z+\frac{1}{y}\right)\left(x+\frac{1}{z}\right)}\ge6\)

\(\Leftrightarrow3\sqrt[3]{\frac{\left(xy+1\right)\left(yz+1\right)\left(xz+1\right)}{xyz}}\ge6\)

Mà \(\frac{z\left(xy+1\right)^2}{y^2\left(yz+1\right)}+\frac{x\left(yz+1\right)^2}{z^2\left(xz+1\right)}+\frac{y\left(xz+1\right)^2}{x^2\left(xy+1\right)}\ge3\sqrt[3]{\frac{\left(xy+1\right)\left(yz+1\right)\left(xz+1\right)}{xyz}}\)

\(\Rightarrow\frac{z\left(xy+1\right)^2}{y^2\left(yz+1\right)}+\frac{x\left(yz+1\right)^2}{z^2\left(xz+1\right)}+\frac{y\left(xz+1\right)^2}{x^2\left(xy+1\right)}\ge6\)

Vậy GTNN của \(\frac{z\left(xy+1\right)^2}{y^2\left(yz+1\right)}+\frac{x\left(yz+1\right)^2}{z^2\left(xz+1\right)}+\frac{y\left(xz+1\right)^2}{x^2\left(xy+1\right)}=6\)

Đặt \(\left(\frac{1}{x};\frac{1}{y};\frac{1}{z}\right)=\left(a;b;c\right)>0\Rightarrow a+b+c=2\)

\(\Rightarrow P=\frac{a^3}{\left(2-a\right)^2}+\frac{b^3}{\left(2-b\right)^2}+\frac{c^3}{\left(2-c\right)^2}\)

Ta có đánh giá: \(\frac{a^3}{\left(2-a\right)^2}\ge\frac{2a-1}{2}\) ; \(\forall a\in\left(0;2\right)\)

Thật vậy, BĐT tương đương:

\(2a^3\ge\left(2a-1\right)\left(a^2-4a+4\right)\)

\(\Leftrightarrow9a^2-12a+4\ge0\Leftrightarrow\left(3a-2\right)^2\ge0\) (luôn đúng)

Tương tự: \(\frac{b^3}{\left(2-b\right)^2}\ge\frac{2b-1}{2}\) ; \(\frac{c^3}{\left(2-c\right)^2}\ge\frac{2c-1}{2}\)

Cộng vế với vế: \(P\ge\frac{2\left(a+b+c\right)-3}{2}=\frac{1}{2}\)

\(P_{min}=\frac{1}{2}\) khi \(a=b=c=\frac{2}{3}\) hay \(x=y=z=\frac{3}{2}\)