Hint: We had to only find the intersection points of the parabola and the line by solving their equations and then applying limits on the integration. We will get the required area because integration is the area under the curve.
Complete step by step solution:
Let us first draw the figure for the parabola \[{y^2} = 4ax\] and the line y = mx.
First, we had to find the coordinates of point A and B. So, for that let us solve the equation of parabola and line.
So, putting the value of y from the equation of line to the equation of parabola.
\[ \Rightarrow {\left( {mx} \right)^2} = 4ax\]
\[ \Rightarrow x\left( {x{m^2} - 4a} \right) = 0\]
So, x = 0, \[\dfrac{{4a}}{{{m^2}}}\]
Now putting the value of x in the equation of line (y = mx).
\[ \Rightarrow y = m \times 0 = 0{\text{ }}or{\text{ }}y = m \times \dfrac{{4a}}{{{m^2}}} = \dfrac{{4a}}{m}\]
So, A(0, 0) and B(\[\dfrac{{4a}}{{{m^2}}}\], \[\dfrac{{4a}}{m}\]) are the point of intersection of line and parabola.
Now we had to find the area of the shaded region in the above figure.
As we can see that in the shaded region parabola is above the line.
So, while applying integration to find the required area we had to subtract the line from the equation of parabola and apply limits from A to B.
So, area = \[\int\limits_A^B {\left( {{y_{parabola}} - {y_{line}}} \right)dx} \]
Now from the equation of parabola we get \[{y_{parabola}} = \pm \sqrt {4ax} \] but as the area is in the first quadrant. So, the \[{y_{parabola}}\] must be positive. Hence, \[{y_{parabola}} = \sqrt {4ax} \]
Area = \[\int\limits_0^{\dfrac{{4a}}{{{m^2}}}} {\left( {\sqrt {4ax} - mx} \right)dx} = \left[ {{{\left( {4a} \right)}^{\dfrac{1}{2}}}\dfrac{{{{\left( x \right)}^{\dfrac{3}{2}}}}}{{\dfrac{3}{2}}} - \dfrac{{m{x^2}}}{2}} \right]_0^{\dfrac{{4a}}{{{m^2}}}}\]
Now putting the upper and lower limits in the above equation.
\[\left[ {{{\left( {4a} \right)}^{\dfrac{1}{2}}}\dfrac{{{{\left( {\dfrac{{4a}}{{{m^2}}}} \right)}^{\dfrac{3}{2}}}}}{{\dfrac{3}{2}}} - \dfrac{{m{{\left( {\dfrac{{4a}}{{{m^2}}}} \right)}^2}}}{2} - \left( {{{\left( {4a} \right)}^{\dfrac{1}{2}}}\dfrac{{{{\left( 0 \right)}^{\dfrac{3}{2}}}}}{{\dfrac{3}{2}}} - \dfrac{{m{{\left( 0 \right)}^2}}}{2}} \right)} \right] = {\left( {4a} \right)^2}\dfrac{2}{{3{m^3}}} - \dfrac{{8{a^2}}}{{{m^3}}} = \dfrac{{32{a^2}}}{{3{m^3}}} - \dfrac{{8{a^2}}}{{{m^3}}} = \dfrac{{8{a^2}}}{{3{m^3}}}\]
So, the area of the shaded region will be \[\dfrac{{8{a^2}}}{{{m^3}}}\].
Hence, the area enclosed between the parabola \[{y^2} = 4ax\] and the line y = mx will be equal to \[\dfrac{{8{a^2}}}{{{m^3}}}\].
Note: Whenever we come up with this type of problem then there is also another method to find the required area. So, for that first we can find the area enclosed by parabola form A to B and then we can find the area of the triangle formed by the line y = mx from A to B with the coordinate axis. After that we can subtract the area of the triangle with the area formed by the parabola to find the required enclosed area.