QJEEAdv24Ma0d09: JEE Advanced 2024 Mathematics - Class 11 JEEAdv24Ma0d09 Solution: Let be such that . Then is equal to :

Given the information, let's start by analyzing the trigonometric relationships involving $\cot x = \frac{-5}{\sqrt{11}}$ where \frac{\pi}{2} < x < \pi .

We know that $\cot x$ is the reciprocal of $\tan x$ . Hence,

$\cot x = \frac{\cos x}{\sin x} = \frac{-5}{\sqrt{11}}$

From the above, it follows that:

$\cos x = -5k \text{ and } \sin x = \sqrt{11}k$

for some constant $k$ . Using the Pythagorean identity:

$\cos^2 x + \sin^2 x = 1$

Let’s substitute the values of $\cos x$ and $\sin x$ into this identity:

$(-5k)^2 + (\sqrt{11}k)^2 = 1$

$25k^2 + 11k^2 = 1$

$36k^2 = 1$

$k^2 = \frac{1}{36}$

$k = \frac{1}{6} \text{ or } k = -\frac{1}{6}$

Therefore, we have two sets of values:

$\cos x = -\frac{5}{6} \text{ and } \sin x = \frac{\sqrt{11}}{6}$

Given that $x$ is in the interval $\left(\frac{\pi}{2}, \pi\right)$ , where sine is positive and cosine is negative, we take:

$\cos x = -\frac{5}{6},\ \sin x = \frac{\sqrt{11}}{6}$

Now consider the expression:

$\left(\sin \frac{11 x}{2}\right)(\sin 6 x - \cos 6 x) + \left(\cos \frac{11 x}{2}\right)(\sin 6 x + \cos 6 x)$

To solve this, first find $\frac{11x}{2}$ which lies in the interval $\left(\frac{11\pi}{4}, \frac{11\pi}{2}\right)$ . Next, we simplify each term using sum-to-product identities:

Notice:

$\sin 6x - \cos 6x = \sqrt{2} \sin \left( 6x - \frac{\pi}{4} \right)$

$\sin 6x + \cos 6x = \sqrt{2} \cos \left( 6x - \frac{\pi}{4} \right)$

Using these, the given expression becomes:

$\left(\sin \frac{11x}{2}\right) \sqrt{2} \sin \left(6x - \frac{\pi}{4} \right) + \left(\cos \frac{11x}{2}\right) \sqrt{2} \cos \left(6x - \frac{\pi}{4} \right)$

This further simplifies using the angle sum identities:

$= \sqrt{2} \left(\sin \frac{11x}{2} \sin \left(6x - \frac{\pi}{4}\right) + \cos \frac{11x}{2} \cos \left(6x - \frac{\pi}{4}\right)\right)$

Utilizing the cosine of the difference identity:

$\sin A \sin B + \cos A \cos B = \cos(A - B)$

Substitute $A = \frac{11x}{2}$ and $B = 6x - \frac{\pi}{4}$ :

$\cos \left(\frac{11x}{2} - \left(6x - \frac{\pi}{4}\right)\right) = \cos \left(\frac{11x}{2} - 6x + \frac{\pi}{4}\right) = \cos \left( - \frac{x}{2} + \frac{\pi}{4} \right) = \cos \left(\frac{\pi}{4} - \frac{x}{2}\right)$

Using the values of trigonometric functions:

$\cos \left(\frac{\pi}{4} - \frac{x}{2}\right) = \frac{\sqrt{11}+1}{2\sqrt{3}}$

Thus, the correct option is:

Option B

$\frac{\sqrt{11}+1}{2 \sqrt{3}}$