General expressions

Although the method is simple to describe, the algebra becomes messy when written in full generality. For example, suppose we use the second method, with three points $(x_1,y_1)$, $(x_2,y_2)$, $(x_3,y_3)$ none of which are at the origin.

The three equations are

$$\begin{array}{cccc}& {\displaystyle (A{x}_1+B{y}_1{)}^2+C{x}_1+D{y}_1=0}& & \\ & {\displaystyle (A{x}_2+B{y}_2{)}^2+C{x}_2+D{y}_2=0}& & \\ & {\displaystyle (A{x}_3+B{y}_3{)}^2+C{x}_3+D{y}_3=0}& & \end{array}$$$$\begin{gather} (Ax_1+By_1)^2+Cx_1+Dy_1=0\\ (Ax_2+By_2)^2+Cx_2+Dy_2=0\\ (Ax_3+By_3)^2+Cx_3+Dy_3=0 \end{gather}$$

Solving the first two for $C$ and $D$ produces:

$$C = \frac{(Ax_2+By_2)^2y_1-(Ax_1+By_1)^2y_2}{x_1y_2-x_2y_1},\quad D = \frac{(Ax_1+By_1)^2x_2-(Ax_2+By_2)^2x_1}{x_1y_2-x_2y_1}$$

It will simplify matters to introduce the notation

$$v_{ij}=x_iy_j-x_jy_i.$$

The discussion at mathpages does much the same thing, but treats the $v$ values as the elements of the cross product of the vectors $[x_1,x_2,x_3]$ and $[y_1,y_2,y_3]$.

Now, substituting into the last equation produces an equation of the form

$$aA^2+2bAB+cB^2=0$$

where

\begin{align*} a & = -v_{23}x_1^2+v_{13}x_2^2-v_{12}x_3^2\\ b & = -v_{23}x_1y_1+v_{13}x_2y_2-v_{12}x_3y_3\\ c & = -v_{23}y_1^2+v_{13}y_2^2-v_{12}y_3^2 \end{align*}

The solutions are then

\begin{align*} A&=r, & B &= \frac{-br+\sqrt{b^2-acr}}{c}\\ A&=s, & B &= \frac{-bs-\sqrt{b^2-acs}}{c} \end{align*}

The values $a$, $b$ and $c$ can all be expressed as the negative determinants:

$$a = -\begin{vmatrix}x_1^2&x_2^2&x_3^2\\ x_1&x_2&x_3\\ y_1&y_2&y_3\end{vmatrix},\qquad b = -\begin{vmatrix}x_1y_1&x_2y_2&x_3y_3\\ x_1&x_2&x_3\\ y_1&y_2&y_3\end{vmatrix},\qquad c = -\begin{vmatrix}y_1^2&y_2^2&y_3^2\\ x_1&x_2&x_3\\ y_1&y_2&y_3\end{vmatrix}.$$

The next step would be to substitute these values into the expressions above for $C$ and $D$, but as you see we're already getting to the reasonable limit of complexity for algebraic expressions. Substituting the first pair of values for $A$ and $B$ into the equation for $C$ produces an utterly hideous expression!