Đặt \(AB=c;BC=a\)

\(S=\frac{1}{2}ah_a=\frac{1}{2}bh_b=\frac{1}{2}ch_c\Rightarrow ah_a=bh_b=ch_c=2S\)

\(\Rightarrow\left\{{}\begin{matrix}h_a=\frac{2S}{a}\\h_b=\frac{2S}{b}\\h_c=\frac{2S}{c}\end{matrix}\right.\) \(\Rightarrow\frac{2S}{b}=\frac{2S}{a}+\frac{2S}{c}\Rightarrow\frac{1}{b}=\frac{1}{a}+\frac{1}{c}\)

\(\Rightarrow\frac{b}{a}+\frac{b}{c}=1\)

\(cosB=\frac{7}{8}=\frac{a^2+c^2-b^2}{2ac}\Leftrightarrow b^2=a^2+c^2-\frac{7}{4}ac\)

\(\Leftrightarrow\left(\frac{a}{b}\right)^2+\left(\frac{c}{b}\right)^2-\frac{7}{4}\left(\frac{a}{b}\right)\left(\frac{c}{b}\right)=1\)

Đặt \(\left\{{}\begin{matrix}\frac{a}{b}=x>0\\\frac{c}{b}=y>0\end{matrix}\right.\) \(\Rightarrow\left\{{}\begin{matrix}\frac{1}{x}+\frac{1}{y}=1\\x^2+y^2-\frac{7}{4}xy=1\end{matrix}\right.\)

\(\Leftrightarrow\left\{{}\begin{matrix}x+y=xy\\x^2+y^2-\frac{7}{4}xy=1\end{matrix}\right.\) \(\Leftrightarrow\left\{{}\begin{matrix}x+y=xy\\\left(x+y\right)^2-\frac{15}{4}xy=1\end{matrix}\right.\)

\(\Leftrightarrow\left(xy\right)^2-\frac{15}{4}xy-1=0\) \(\Rightarrow\left[{}\begin{matrix}xy=4\\xy=-\frac{1}{4}\left(l\right)\end{matrix}\right.\)

\(\Rightarrow\left\{{}\begin{matrix}x+y=4\\xy=4\end{matrix}\right.\) \(\Rightarrow x=y=2\)

\(\Rightarrow\left\{{}\begin{matrix}\frac{a}{b}=2\\\frac{c}{b}=2\end{matrix}\right.\) \(\Rightarrow a=c=2b\)

\(\Rightarrow p=\frac{a+b+c}{2}=\frac{5b}{2}\) \(\Rightarrow S=\sqrt{p\left(p-b\right)\left(p-2b\right)\left(p-2b\right)}=\frac{b^2\sqrt{15}}{4}\)