Fekete-Szegö inequality of bi-starlike and bi-convex functions of order $b$ associated with symmetric $q$-derivative in conic domains

Theorem 1. If $f \in k - ST_{\Sigma, b}(\alpha, \beta)$ and is of the form (1.1) then $$|a_2| \leq \frac{P_1 \sqrt{P_1} b^2}{\sqrt{[P_1^2 b(\widetilde{[3]}_q - \widetilde{[2]}_q) + 2(P_1 - P_2)(\widetilde{[2]}_q - 1)^2]}}, \quad |a_3| \qquad \leq \frac{b^2 P_1^2}{(\widetilde{[2]}_q - 1)^2} + \frac{b P_1}{(\widetilde{[3]}_q - 1)}$$ and $$|a_3 - \mu a_2^2| \le \begin{cases} \frac{P_1 b}{(\widetilde{[3]}_q - 1)}, & \text{if } 0 \le |s(\mu)| \le 1\\ \frac{P_1 b |s(\mu)|}{(\widetilde{[3]}_q - 1)} & \text{if } |s(\mu)| \ge 1, \end{cases}$$ where $$s(\mu) = \frac{P_1^2 b(1-\mu)}{4[P_1^2 b(\widetilde{[3]}_q - \widetilde{[2]}_q) + 2(P_1 - P_2)(\widetilde{[2]}_q - 1)^2]}.$$

Theorem 2. If $f \in k - UCV_{\Sigma, b}(\alpha, \beta)$ and is of the form (1.1) then $$|a_2| \leq \frac{P_1\sqrt{P_1b}}{\sqrt{\left[2\widetilde{[2]}_q(\widetilde{[3]}_q - \widetilde{[2]}_q)bP_1^2 + \widetilde{[2]}_q^2(P_1 - P_2)\right]}} \ and \ |a_3| \leq \frac{P_1^2b^2}{\widetilde{[2]}_q^2} + \frac{bP_1}{\widetilde{[2]}_q\widetilde{[3]}_q}$$ and $$|a_3 - \mu a_2^2| \le \begin{cases} \frac{P_1 b}{\widetilde{[2]}_q \widetilde{[3]}_q}, & \text{if } 0 \le |s(\mu)| \le 1\\ \frac{P_1 b |s(\mu)|}{\widetilde{[2]}_q \widetilde{[3]}_q} & \text{if } |s(\mu)| \ge 1, \end{cases}$$ where $$s(\mu) = \frac{P_1^2 b(1-\mu)}{4 \left[ 2\widetilde{[2]}_q(\widetilde{[3]}_q - \widetilde{[2]}_q) b P_1^2 + \widetilde{[2]}_q^2 (P_1 - P_2) \right]} \ .$$

Definition 1. A function $f \in \mathcal{A}$ is said to be in the class $k - ST(\alpha, \beta)$ if and only if $$\Re\left\{\frac{zf'(z)}{f(z)}\right\} - \beta > k \left|\frac{zf'(z)}{f(z)} - \alpha\right|, \ z \in \Delta,\tag{1.4}$$ where $0 \le \beta < \alpha \le 1$ and $k(1 - \alpha) < 1 - \beta$ .

Definition 2. A function $f \in \mathcal{A}$ is said to be in the class $k - UCV(\alpha, \beta)$ if and only if $$\Re\left(1 + \frac{zf''(z)}{f'(z)}\right) - \beta > k \left| 1 + \frac{zf''(z)}{f'(z)} - \alpha \right|, \ z \in \Delta.$$ $$\tag{1.5}$$ where $0 \le \beta < \alpha \le 1$ and $k(1 - \alpha) < 1 - \beta$ . In particular, for $\alpha=1$ , $\beta=0$ the classes $k-ST(\alpha,\beta)$ and $k-UCV(\alpha,\beta)$ reduces to k-ST and k-UCV respectively. Further, for $\alpha=1$ these classes coincides with the classes studied by Nishiwaki et al. [18] and Shams et al. [23]. In 2017, Annamalai et al. [5], obtained second Hankel determinant of analytic functions involving conic domains. Geometric Interpretation: A function $f \in k - ST(\alpha, \beta)$ and $k - UCV(\alpha, \beta)$ if and only if $\frac{zf'(z)}{f(z)}$ and $1 + \frac{zf''(z)}{f'(z)}$ , respectively takes all the values in the conic domain $\Omega_{k,\alpha,\beta}$ . $$\Omega_{k, \alpha, \beta} = \{ \omega : \omega \in \mathbb{C} \text{ and } k | \omega - \alpha | < \Re(\omega) - \beta \}$$ or $$\Omega_{k, \alpha, \beta} = \{ \omega : \omega \in \mathbb{C} \text{ and } k\sqrt{[\Re(\omega) - \alpha]^2 + [\Im(\omega)]^2} < \Re(\omega) - \beta \},$$ where $0 \le \beta < \alpha \le 1$ and $k(1 - \alpha) < 1 - \beta$ . Clearly $1 \in \Omega_{k, \alpha, \beta}$ and $\Omega_{k, \alpha, \beta}$ is bounded by the curve $$\partial\Omega_{k,\alpha,\beta} = \{\omega : \omega = u + iv \text{ and } k^2(u-\alpha)^2 + k^2v^2 = (u-\beta)^2\}.$$

Definition 3. The Caratheodory functions $p \in P$ is said to be in the class $\mathcal{P}(p_{k,\alpha,\beta})$ if and only if p takes all the values in the conic domain $\Omega_{k,\alpha,\beta}$ . Analytically it is defined as follows: $$\mathcal{P}(p_{k,\,\alpha,\,\beta}) = \{ p : \ p \in \mathcal{P} \text{ and } p(\Delta) \subset \Omega_{k,\,\alpha,\,\beta} \},$$ $$\mathcal{P}(p_{k,\,\alpha,\,\beta}) = \{ p : \ p \in \mathcal{P} \text{ and } p(z) \prec p_{k,\,\alpha,\,\beta}, \ z \in \Delta \}.$$ Note that $\partial\Omega_{k,\alpha,\beta}$ represents conic section about real axis. In particular, $\Omega_{k,\alpha,\beta}$ represents an elliptic domain for k > 1, parabolic domain for for k = 1, hyperbolic domain for 0 < k < 1. Sim et al. [24] obtained the functions $p_{k,\alpha\beta}(z)$ which play the role of extremal functions of $\mathcal{P}(p_{k,\alpha,\beta})$ as $$p_{k,\alpha\beta}(z) = \begin{cases} \frac{1 + (1 - 2\beta)z}{1 - z}, & \text{for } k = 0; \\ \alpha + \frac{2(\alpha - \beta)}{\pi^2} \log^2 \left(\frac{1 + \sqrt{u_k(z)}}{1 - \sqrt{u_k(z)}}\right), & \text{for } k = 1; \\ \frac{(\alpha - \beta)}{1 - k^2} \cosh\{\mathfrak{u}(k)\log\left(\frac{1 + \sqrt{u_k(z)}}{1 - \sqrt{u_k(z)}}\right)\} + \frac{\beta - \alpha k^2}{1 - k^2}, & \text{for } 0 < k < 1; \\ \frac{(\alpha - \beta)}{k^2 - 1} \sin^2 \left(\frac{\pi}{2K(k)} \int_0^\omega \frac{dt}{\sqrt{1 - t^2}\sqrt{1 - t^2k^2}}\right) + \frac{\alpha k^2 - \beta}{k^2 - 1}, & \text{for } k > 1; \end{cases}$$ where $u(k) = \frac{2}{\pi} \cos^{-1} k$ , $u_k(z) = \frac{z + \rho_k}{1 + \rho_k z}$ and $$\rho_{k} = \begin{cases} \left(\frac{e^{A} - 1}{e^{A} + 1}\right)^{2}, & \text{for } k = 1; \\ \left(\frac{\exp\left(\frac{1}{u_{k}(z)}\operatorname{arc \, cosh} B\right) - 1}{\exp\left(\frac{1}{u_{k}(z)}\operatorname{arc \, cosh} B\right) + 1}\right)^{2}, & \text{for } 0 < k < 1; \\ \sqrt{k}\sin\left[\frac{2K(\kappa)}{\pi}\operatorname{arc \, sin} C\right], & \text{for } k > 1; \end{cases}$$ with $A = \sqrt{\frac{1-\alpha}{2(\alpha-\beta)}\pi}$ , $B = \frac{1}{\alpha-\beta}(1-k^2-\beta+\alpha k^2)$ , $C = \frac{1}{\alpha-\beta}(k^2-1+\beta-\alpha k^2)$ . $$\begin{split} K(\kappa) &= \int_0^\omega \frac{dt}{\sqrt{1-t^2}\sqrt{1-t^2\kappa^2}} \ (0<\kappa<1), \\ K'(\kappa) &= K(\sqrt{1-\kappa^2}) \ (0<\kappa<1), \\ \kappa &= \cosh\Bigl(\frac{\pi K'(\kappa)}{4K(\kappa)}\Bigr). \end{split}$$ According to Koebe's $\frac{1}{4}$ theorem, every analytic and univalent function f in $\Delta$ has an inverse $f^{-1}$ and is defined as $$f^{-1}(f(z)) = z, (z \in \Delta), f(f^{-1}(w)) = w(|w| < r_0(f); r_0(f) \ge \frac{1}{4}).$$ Also the function $f^{-1}$ can be written as $$f^{-1}(w) = w - a_2 w^2 + (2a_2^2 - a_3)w^3 - (5a_2^3 - 5a_2 a_3 + a_4)w^4 + \dots$$ (1.6)

Definition 4. A function $f \in \mathcal{A}$ is said to be bi-univalent if both f and analytic extension of $f^{-1}$ in $\Delta$ are univalent in $\Delta$ . The class of all bi-univalent functions is denoted by $\Sigma$ . That is a function f is said to be bi-univalent if and only if - (1) f is an analytic and univalent function in $\Delta$ . - (2) There exists an analytic and univalent function g in $\Delta$ such that f(q(z)) = q(f(z)) = z in $\Delta$ . The class of bi-univalent functions was introduced by Lewin [15] in 1967. Recently many researchers ([1], [4], [12], [19], [20], [21], [25], [26], [27], [28], [29], [30], [31], [32]) have introduced and investigated several interesting subclasses of the bi-univalent functions and they have found non-sharp estimates of two Taylor-Maclaurin coefficients $|a_2|$ , $|a_3|$ , Fekete-Szegö inequality and second Hankel determinants. In 2017, Şahsene Altinkaya, Sibel Yalçin [2], [3] estimated the coefficients and Fekete-Szegö inequality for some subclasses of bi-univalent functions involving symmetric q-derivative operator subordinate to the generating function of Chebyshev polynomial.

Definition 5. [11] Jackson defined q-derivative operator $D_q$ of an analytic function f of the form (1.1)as follows: $$D_q f(z) = \begin{cases} \frac{f(qz) - f(z)}{(q-1)z}, & for \ z \neq 0, \\ f'(0), & for \ z = 0 \end{cases}$$ $$D_q f(0) = f'(0) \text{ and } D_q^2 = D_q(D_q f(z)).$$ If $f(z) = z^n$ for any positive integer n, the q-derivative of f(z) is defined by $$D_q z^n = \frac{(qz)^n - z^n}{qz - z} = [n]_q z^{n-1},$$ where $[n]_q = \frac{q^n - 1}{q - 1}$ . As $q \to 1^-$ and $k \in \mathbb{N}$ , we have $[n]_q \to n$ and $\lim_{q \to 1} (D_q f(z)) = f'(z)$ where f' is normal derivative of f. Therefore $$D_q f(z) = 1 + \sum_{n=2}^{\infty} [n]_q a_n z^{n-1}.$$

Definition 6. [7] The symmetric q-derivative operator $\widetilde{D}_q$ of an analytic function f is defined as follows: $$(\widetilde{D}_q f)(z) = \begin{cases} \frac{f(qz) - f(q^{-1}z)}{(q - q^{-1})z}, & \text{for } z \neq 0, \\ f'(0), & \text{for } z = 0 \end{cases}.$$ It is clear that $\widetilde{D}_q z^n = [\widetilde{n}]_q z^{n-1}$ and $\widetilde{D}_q f(z) = 1 + \sum_{n=2}^{\infty} [\widetilde{n}]_q a_n z^{n-1}$ , where $[\widetilde{n}]_q = \frac{q^n - q^{-n}}{q - q^{-1}}$ . The relation between q-derivative operator and symmetric q-derivative operator is given by $$(\widetilde{D_q}f)(z) = D_{q^2}f(q^{-1}z).$$ If g is the inverse of f then $$(\widetilde{D}_q g)(w) = \frac{g(qw) - g(q^{-1}w)}{(q - q^{-1})w}$$ $$= 1 - [\widetilde{2}]_q a_2 w + [\widetilde{3}]_q (2a_2^2 - a_3) w^2 - [\widetilde{4}]_q (5a_2^3 - 5a_2 a_3 + a_4) w^3 + \dots$$ The q-calculus has so many applications in various branches of mathematics and physics. Jackson [11] developed q-integral and q-derivative in a systematic way. The fractional q-calculus is an important tool used to study various families of analytic functions. In recent years, several subclasses of analytic functions involving fractional q-integral and fractional q-derivative operators were constructed and coefficient inequality, Fekete-Szegö inequality and Hankel determinant were estimated for the functions in these classes. Motivated by the above mentioned work, in this paper, bi-starlike functions of order b and bi-convex functions of order b involving q-derivative operator subordinate to the conic domains are defined and the Fekete-Szeg $\ddot{o}$ inequality for the function in these classes are obtained.

Definition 7. A function $f \in \Sigma$ is said to be in the class $k - ST_{\Sigma, b}(\alpha, \beta)$ ; where $0 \le \beta < \alpha \le 1$ and $k(1-\alpha) < 1-\beta$ , and b is a non-zero complex number, if it satisfies the following conditions: $$1 + \frac{1}{b} \left[ \frac{z \widetilde{D}_q f(z)}{f(z)} - 1 \right] \prec p_{k, \alpha, \beta}(z) \quad \text{and} \quad 1 + \frac{1}{b} \left[ \frac{w \widetilde{D}_q g(w)}{g(w)} - 1 \right] \prec p_{k, \alpha, \beta}(w) \quad (1.7)$$ where g is an extension of $f^{-1}$ to $\Delta$ .

Definition 8. A function $f \in \Sigma$ is said to be in the class $k - UCV_{\Sigma, b}(\alpha, \beta)$ ; where $0 \le \beta < \alpha \le 1$ and $k(1 - \alpha) < 1 - \beta$ , and b is a non-zero complex number, if it satisfies the following conditions: $$1 + \frac{1}{b} \left[ \frac{z \widetilde{D}_q(\widetilde{D}_q f(z))}{\widetilde{D}_q(f(z))} \right] \prec p_{k, \alpha, \beta}(z) \quad \text{and} \quad 1 + \frac{1}{b} \left[ \frac{w \widetilde{D}_q(\widetilde{D}_q g(w))}{\widetilde{D}_q(g(w))} \right] \prec p_{k, \alpha, \beta}(w) \quad (1.8)$$ where g is an extension of $f^{-1}$ to $\Delta$ .