amp;a=2i^−j^+3k^amp; b=3i^−5j^+k^amp;a×c=c×bamp;a×c+b×c=0amp;(a+b)×c=0amp;⇒c=λ(a+b)amp;c=λ(5i^−6j^+4k^)…..(1)amp;∣c∣2=λ2(25+36+16)amp;∣c∣2=77λ2amp;(a+c)⋅(b+c)=168amp;a⋅ b+a⋅c+c⋅ b+∣c∣2=168amp;14+c⋅(a+b)+77λ2=168
using equation (1)
amp;λ∣5i^−6j^+4k^∣2+77λ2=154amp;77λ+77λ2−154=0amp;λ2+λ−2=0amp;λ=−2,1
∴ Maximum value of ∣c∣2 occurs when λ=−2
amp;∣c∣2=77λ2amp;=77×4amp;=308