Sharp Coefficient Estimates for Analytic Functions Subordinate to the Cusp Domain: Theory and Image Processing Applications

Lemma 1. (See [49,50]). Let $p \in \mathcal{P}$ with expansion <span id="page-7-3"></span><span id="page-7-0"></span> $$p(z) = 1 + \sum_{k=1}^{\infty} p_k z^k.$$ (3) Then the following bounds hold: <span id="page-7-2"></span><span id="page-7-1"></span> $$|p_k| \le 2, \quad k \ge 1,\tag{4}$$ $$|p_{k+n} - \mu p_k p_n| < 2, \quad 0 < \mu \le 1,$$ (5) $$|p_2 - \eta p_1^2| < 2 \max\{1, |2\eta - 1|\}, \quad \eta \in \mathbb{C}.$$ (6) Inequalities (4) and (5) are from [49], whereas (6) is from [50].

Lemma 2. (See [51]). For $p \in \mathcal{P}$ , as in (3), if $R \in [0,1]$ satisfies $R(2R-1) \leq S \leq R$ , then $$|Sp_3 - 2Rp_1p_2 + p_3| \le 2.$$

Lemma 3. (See [52]). Let $\xi$ , $\zeta$ , $\mu$ , $\lambda \in (0,1)$ satisfy $$8\lambda(1-\lambda)\left[(\xi\zeta-2\mu)^2+(\xi(\lambda+\xi)-\zeta)^2\right]+\xi(1-\xi)(\zeta-2\lambda\xi)^2\leq 4\lambda\xi^2(1-\xi)^2(1-\lambda).$$ Then for $p \in \mathcal{P}$ , as in (3), $$|\mu p_4 + \lambda p_2^2 + 2\xi p_1 p_3 - \frac{3}{2}\xi p_1^2 p_2 - p_4| \le 2.$$

Lemma 4. (See [6,52]). Let $p \in \mathcal{P}$ have the form (3), and let $x, z \in \overline{\mathcal{U}}$ . Then $$\begin{aligned} 2p_2 &= p_1^2 + x(4 - p_1^2), \\ 4p_3 &= 2x(4 - p_1^2)p_1 - x^2(4 - p_1^2)p_1 + 2z(1 - |x|^2)(4 - p_1^2) + p_1^3. \end{aligned}$$ These lemmas form the foundational tools for deriving coefficient bounds and Hankel determinant estimates in the subsequent analysis.

Theorem 1. Let f ∈ Mtan be given by [\(1\)](#page-1-0). Then the coefficients satisfy $$|a_2| \le \frac{3}{4},\tag{7}$$ <span id="page-8-6"></span> $$|a_3| \le \frac{1}{2},\tag{8}$$ $$|a_4| \le \frac{3}{8},\tag{9}$$ <span id="page-8-7"></span> $$|a_5| \le \frac{3}{10}. (10)$$ These bounds are sharp. Sharpness is attained by the functions $$\int_0^z \frac{2(1+\arctan t)}{1+e^{-t}} dt = z + \frac{3}{4}z^2 + \frac{1}{6}z^3 - \frac{3}{32}z^4 - \frac{1}{24}z^5 + \frac{49}{1440}z^6 + \dots, \tag{11}$$ <span id="page-8-3"></span> $$\int_0^z \frac{2(1+\arctan(t^2))}{1+e^{-t^2}} dt = z + \frac{1}{2}z^3 + \frac{1}{10}z^5 + \dots,$$ (12) <span id="page-8-5"></span> $$\int_0^z \frac{2(1+\arctan(t^3))}{1+e^{-t^3}} dt = z + \frac{3}{8}z^4 + \dots,$$ (13) <span id="page-8-4"></span> $$\int_0^z \frac{2(1+\arctan(t^4))}{1+e^{-t^4}} dt = z + \frac{3}{10}z^5 + \dots$$ (14) Proof. Assume f ∈ Mtan. By definition, we have <span id="page-8-0"></span> $$f'(z) = \frac{2(1 + \arctan w(z))}{1 + e^{-w(z)}},\tag{15}$$ where w is a Schwarz function with |w(z)| < 1 in U. Define $$p(z) = \frac{1+w(z)}{1-w(z)} = 1 + p_1 z + p_2 z^2 + p_3 z^3 + \cdots$$ so that p ∈ P, the class of Carathéodory functions, and $$w(z) = \frac{p(z) - 1}{p(z) + 1} = \frac{p_1 z + p_2 z^2 + p_3 z^3 + \cdots}{2 + p_1 z + p_2 z^2 + \cdots}.$$ Expanding f ′ (z) in powers of z gives <span id="page-8-2"></span><span id="page-8-1"></span> $$f'(z) = 1 + 2a_2z + 3a_3z^2 + 4a_4z^3 + 5a_5z^4 + \cdots,$$ (16) while the right-hand side of [\(15\)](#page-8-0) expands as $$\frac{2(1 + \arctan w(z))}{1 + e^{-w(z)}} = 1 + \frac{3}{4}p_1z - \frac{1}{4}(p_1^2 - 3p_2)z^2 + \frac{1}{64}(p_1^3 - 32p_1p_2 + 48p_3)z^3 + (\frac{11}{192}p_1^4 + \frac{3}{64}p_1^2p_2 - \frac{1}{2}p_1p_3 - \frac{1}{4}p_2^2 + \frac{3}{4}p_4)z^4 + \cdots$$ (17) Comparing the coefficients in (16) and (17), we find that <span id="page-9-1"></span><span id="page-9-0"></span> $$a_2 = \frac{3p_1}{8},\tag{18}$$ $$a_3 = \frac{-p_1^2 + 3p_2}{12},\tag{19}$$ $$a_4 = \frac{p_1^3 - 32p_1p_2 + 48p_3}{256},\tag{20}$$ <span id="page-9-3"></span><span id="page-9-2"></span> $$a_5 = \frac{11p_1^4 + 9p_1^2p_2 - 48p_2^2 - 96p_1p_3 + 144p_4}{960}. (21)$$ Applying Lemma 1 together with the triangle inequality, we systematically estimate the initial coefficients: $$a_2 = \frac{3p_1}{8}$$ , implies $|a_2| \le \frac{3}{4}$ , $a_3 = \frac{1}{12}(3p_2 - p_1^2) = \frac{1}{4}(p_2 - \frac{1}{3}p_1^2)$ , implies $|a_3| \le \frac{1}{2}$ , where the bound for $a_3$ follows from inequality (5) in Lemma 1. Similarly, $$a_4 = \frac{1}{256} (p_1^3 - 32p_1p_2 + 48p_3) = \frac{3}{16} \left( \frac{1}{48} p_1^3 - \frac{2}{3} p_1 p_2 + p_3 \right),$$ $$|a_4| = \frac{3}{16} \left| \frac{1}{48} p_1^3 - \frac{2}{3} p_1 p_2 + p_3 \right| \le \frac{3}{8},$$ where the bound for $a_4$ is obtained by applying Lemma 2 with parameters $$R \in [0,1], \quad R(2R-1) = -\frac{1}{9} \le S = \frac{1}{48} \le R = \frac{1}{3}.$$ Similarly, $$a_5 = \frac{1}{5} \left( \frac{11}{192} p_1^4 + \frac{3}{64} p_1^2 p_2 - \frac{1}{4} p_2^2 - \frac{1}{2} p_1 p_3 + \frac{3}{4} p_4 \right),$$ $$|a_5| = \frac{3}{20} \left| -\frac{11}{144} p_1^4 - \frac{1}{16} p_1^2 p_2 + \frac{1}{3} p_2^2 + \frac{2}{3} p_1 p_3 - p_4 \right| \le \frac{3}{10},$$ where the bound for $a_5$ follows by applying Lemma 3 with parameters $$\lambda = \frac{1}{3}$$ , $\xi = \frac{1}{3}$ , $\zeta = \frac{3}{8}$ , $\mu = -\frac{11}{144}$ , and verifying the condition $$\begin{split} &8\lambda(1-\lambda)\left[(\xi\zeta-2\mu)^2+(\xi(\lambda+\xi)-\zeta)^2\right]+\xi(1-\xi)(\zeta-2\lambda\xi)^2\\ &=0.04286\leq 0.04389\\ &=4\lambda\xi^2(1-\xi)^2(1-\lambda). \end{split}$$ The determination of sharp coefficient bounds in the theorem above naturally leads to the following conjecture, which is expected to provide a unifying principle for general coefficient estimates. Conjecture. Let $f \in \mathbb{M}_{tan}$ be given by (1). Then the sharp general coefficient bound is $$|a_k| \leq \frac{3}{2k}, \quad k = 2, 3, \dots.$$ 4.2. Sharp Estimates for the Zalcman Functional in M_tan In the 1960s, Zalcman proposed a conjecture for functions in the class S, given by (1), stating that $$|\mathcal{Z}_n(f)| := |a_n^2 - a_{2n-1}| \le (n-1)^2, \quad n \ge 2,$$ with equality only for the Koebe function $k(z) = \frac{z}{(1-z)^2}$ and its rotations. As noted in [53,54], this also implies the Bieberbach conjecture, $|a_n| \le n$ for $n \ge 2$ . For n=2, the inequality follows directly from the Area Theorem. Recently, the Zalcman functional has attracted attention [49,50,55], particularly the inequalities $|a_3-a_2^2|$ , derivable from the Fekete–Szegö inequality with $\eta=1$ , and $|a_3^2-a_5|$ , for which we identify extremal functions attaining equality. Here, we focus on n = 3 and, for the class $M_{tan}$ , establish explicit upper bounds for the Zalcman functional, confirming the conjecture in this context.

Theorem 2. Let $f \in \mathbb{M}_{tan}$ . Then, for any complex number $\eta$ , we have <span id="page-10-0"></span> $$|a_3 - \eta a_2^2| \le \max\left\{\frac{1}{2}, \frac{|27\eta - 8|}{48}\right\}.$$ (22) Proof. From Equations (18) and (19), we can write $$a_3 - \eta a_2^2 = \frac{1}{12} \left( 3p_2 - p_1^2 \right) - \frac{9\eta}{64} p_1^2 = \frac{1}{4} \left| p_2 - \left( \frac{16 + 27\eta}{48} \right) p_1^2 \right|.$$ Applying inequality (6) from Lemma 1, we get $$|a_3 - \eta a_2^2| \le \frac{1}{2} \max \left\{ 1, \frac{|27\eta - 8|}{24} \right\}.$$ Simplifying the expression gives the stated bound (22). $\Box$

Corollary 1. For any $f \in \mathbb{M}_{tan}$ , we have <span id="page-10-1"></span> $$|\mathcal{H}_{2,1}(f)| \le \max\left\{\frac{1}{2}, \frac{19}{48}\right\} = \frac{1}{2}.$$ (23) This bound is sharp, and equality is attained by the extremal function in (12).

Theorem 3. Let $f \in M_{tan}$ . Then, the Zalcman functional for the third coefficient satisfies the sharp bound $$|a_5 - a_3^2| \le \frac{3}{10}. (24)$$ Equality is attained by (14), showing that the relation between the third and fifth coefficients in $\mathbb{M}_{tan}$ is sharp. Proof. From Equations (19) and (21), we have $$a_5 - a_3^2 = \frac{13}{2880}p_1^4 + \frac{49}{960}p_1^2p_2 - \frac{9}{80}p_2^2 - \frac{1}{10}p_1p_3 + \frac{3}{20}p_4p_4$$ which gives $$|a_5 - a_3^2| = \frac{3}{20} \left| -\frac{13}{432} p_1^4 - \frac{49}{144} p_1^2 p_2 + \frac{3}{4} p_2^2 + \frac{2}{3} p_1 p_3 - p_4 \right|.$$ Applying Lemma 3 with $$\lambda = \frac{3}{4}$$ , $\xi = \frac{1}{3}$ , $\zeta = \frac{49}{126}$ , $\mu = -\frac{13}{432}$ and verifying the required condition, we obtain the sharp bound $$|a_5 - a_3^2| \le \frac{3}{10}.$$ 4.3. Krushkal Inequality for the Class M_tan In this subsection, we aim to establish the following inequality: $$|\mathcal{K}_{n,m}(f)| := \left| a_n^m - a_2^{m(n-1)} \right| \le 2^{m(n-1)} - n^m,$$ for functions of the form (1). Particular attention is devoted to the cases n = 4, m = 1 and n = 5, m = 1, which are of special significance in this context.

Theorem 4. Let $f \in \mathbb{M}_{tan}$ . Then the following inequality holds: $$\left| a_4 - a_2^3 \right| \le \frac{3}{8}.\tag{25}$$ This inequality is sharp by (13). Proof. We prove the inequality in three steps. First, from Equations (18) and (20), we have $$a_4 - a_2^3 = \frac{3}{16} \left( -\frac{25}{96} p_1^3 - \frac{2}{3} p_1 p_2 + p_3 \right).$$ Second, taking absolute values and applying Lemma 2 with parameters satisfying $$R \in [0,1], \quad R(2R-1) \le S = \frac{25}{96} \le R = \frac{1}{3},$$ we get $$\left|a_4 - a_2^3\right| = \frac{3}{16} \left| -\frac{25}{96} p_1^3 - \frac{2}{3} p_1 p_2 + p_3 \right| \le \frac{3}{16} \cdot 2.$$ Finally, this yields $$\left|a_4 - a_2^3\right| \le \frac{3}{8}.$$

Theorem 5. Let $f \in M_{tan}$ . Then $$\left| a_5 - a_2^4 \right| \le \frac{2}{15}.\tag{26}$$ This bound is sharp. Proof. From Equations (19) and (21), we have $$a_5 - a_2^4 = \frac{-511}{61440}p_1^4 + \frac{3}{320}p_1^2p_2 - \frac{1}{20}p_2^2 - \frac{1}{10}p_1p_3 + \frac{3}{20}p_4,$$ which gives $$|a_5 - a_2^4| = \frac{2}{30} \left| \frac{511}{9216} p_1^4 - \frac{1}{16} p_1^2 p_2 + \frac{1}{3} p_2^2 + \frac{2}{3} p_1 p_3 - p_4 \right|.$$ Applying Lemma 3 with $$\lambda = \xi = \frac{1}{2}$$ , $\zeta = \frac{1}{16}$ , $\mu = \frac{511}{6144}$ and verifying the required condition, we obtain the sharp bound $$|a_5 - a_2^4| \le \frac{2}{15}.$$

Theorem 6. Let $f \in \mathbb{M}_{tan}$ . Then the following inequality holds: <span id="page-12-0"></span> $$|a_2a_3 - a_4| \le \frac{3}{8}. (27)$$ The inequality is sharp by (13). Proof. The proof can be structured in three main steps: Step 1: Express the combination of coefficients. From the coefficient formulas, we have $$a_2a_3 - a_4 = \frac{1}{256} (-9p_1^3 + 56p_1p_2 - 48p_3).$$ Step 2: Rewrite in a convenient form for estimation. This can be rearranged as $$|a_2a_3 - a_4| = \frac{3}{16} \left| \frac{3}{16} p_1^3 - \frac{7}{6} p_1 p_2 + p_3 \right|.$$ Step 3: Apply Lemma 2 to bound the expression. Using the lemma with parameters satisfying $$R \in [0,1], \quad R(2R-1) \le S = \frac{3}{16} \le R = \frac{7}{12},$$ we obtain $$|a_2a_3-a_4|\leq \frac{3}{8}$$ Hence, the proof is complete. $\Box$

Theorem 7. Let $f \in \mathbb{M}_{tan}$ . Then the following inequality holds: <span id="page-12-1"></span> $$|a_2a_4 - a_3^2| \le \frac{1}{4}. (28)$$ the upper bound is sharp by (12). Proof. From the coefficient relations, we have $$a_2a_4 - a_3^2 = -\frac{101}{18432}p_1^4 - \frac{1}{192}p_1^2p_2 - \frac{1}{16}p_2^2 + \frac{9}{128}p_1p_3.$$ Taking $p_1 = p$ , |z| = 1, and y = |x| in Lemma 4, we obtain $$|a_2a_4 - a_3^2| = \left| -\frac{113}{18432}p^4 - \frac{9}{512}(4 - p^2)p^2x^2 + \frac{1}{768}(4 - p^2)p^2x^2 + \frac{1}{768}(4 - p^2)p^2x^2 + \frac{1}{54}p(4 - p^2)(1 - |x|^2)z - \frac{1}{64}(4 - p^2)^2x^2 \right|$$ $$\leq \frac{113}{18432}p^4 + \frac{9}{512}(4 - p^2)p^2y^2 + \frac{1}{768}(4 - p^2)p^2y^2 + \frac{1}{54}p(4 - p^2)(1 - y^2) + \frac{1}{64}(4 - p^2)^2y^2$$ $$:= E(y, p).$$ Our next objective is to determine the maximum of E(y, p). Accordingly, $$E_y = \frac{18}{512}(4 - p^2)p^2y + \frac{1}{768}(4 - p^2)p^2 - \frac{2}{54}p(4 - p^2)y + \frac{2}{64}(4 - p^2)^2y$$ $$= \frac{1}{6912}(4 - p^2)(864y + 27yp^2 - 256py + 9p^2).$$ It is evident that $E_y \ge 0$ for all $y \in [0,1]$ . Consequently, the maximum of E(y,p) occurs at y = 1, resulting in $$E(p) := E(1, p) = \frac{113}{18432}p^4 + \frac{29}{1536}(4 - p^2)p^2 + \frac{1}{64}(4 - p^2)^2.$$ Next, we determine the value of p that maximizes E(p). Accordingly, $$E'(p) = \frac{1}{4608}p(-456 + 53p^2).$$ Setting the derivative to zero, we find the three roots: $$p = 0$$ , $p = -2\sqrt{\frac{114}{53}} \approx -2.9$ , $p = 2\sqrt{\frac{114}{53}} \approx 2.9$ . As the only root within the interval [0,2] is p=0, and since E''(0)<0, the maximum is achieved at p=0, which yields $$|a_2a_4 - a_3^2| \le E(1,0) = \frac{1}{4}.$$ Therefore, the proof is complete, confirming the validity of the result. $\Box$

Theorem 8. Let $f \in A$ be given as in (1). If $f \in M_{tan}$ , then $$|\mathcal{H}_{3,1}(f)| \le \frac{133}{320}.$$ Moreover, this bound is the best possible. Proof. Applying the triangle inequality, we have $$|\mathcal{H}_{3,1}(f)| \le |a_3||a_2a_4 - a_3^2| + |a_4||a_4 - a_2a_3| + |a_5||a_3 - a_2^2|.$$ Substituting the estimates from (8)–(10), (23), (27), and (28) yields $$|\mathcal{H}_{3,1}(f)| \leq \left(\frac{1}{2}\right) \left(\frac{1}{4}\right) + \left(\frac{3}{8}\right) \left(\frac{3}{8}\right) + \left(\frac{3}{10}\right) \left(\frac{1}{2}\right).$$ After straightforward simplification, we obtain the desired inequality. Hence, the proof is complete. $\Box$ Although we have derived sharp bounds for all coefficient inequalities, the theorem above provides the best possible estimate for the Hankel determinant, but not necessarily the sharp bound. Nevertheless, after testing several examples of functions belonging to this class, including those generated by the cusp function h(z), we observed that in all cases the determinant did not exceed the value $\frac{9}{64}$ , which was precisely attained by the cusp function itself. This motivated the following conjecture, which we leave as an open problem for further investigation. Conjecture. Let $f \in \mathcal{A}$ be given as in (1). If $f \in \mathbb{M}_{tan}$ , then $$|\mathcal{H}_{3,1}(f)| \le \frac{9}{64}.$$ This bound is sharp.

Definition 1. A function f ∈ Mtan, as in [\(1\)](#page-1-0), belongs to this class if $$f'(z) \prec \frac{2(1+\arctan z)}{1+e^{-z}}, \quad z \in \mathcal{U}.$$ (2) Example 1. The class Mtan is nonempty and, in fact, contains infinitely many analytic functions. We list below some illustrative examples: Mathematics 2026, 14, 1075 7 of 22 1. Consider $$f_{\varrho}(z) = \frac{z}{1 - \varrho z}, \quad |\varrho| < 1.$$ Then f^ϱ ∈ Mtan when |ϱ| ≤ 0.38, since $$f'(z) = \frac{1}{(1 - \varrho z)^2} \prec h(z), \quad |\varrho| \le 0.38.$$ 2. Since $$\frac{1}{9} + \frac{8}{9} \exp\left(\frac{8}{9}z\right) \prec h(z),$$ it follows that $$g(z) = \frac{1}{9}z + \exp\left(\frac{8}{9}z\right) - 1 \in \mathbb{M}_{tan}.$$ 3. Let $$i(z) = z + \frac{1}{2}z^2 - \frac{1}{12}z^4.$$ Then i ∈ Mtan because $$i'(z) = 1 + z - \frac{1}{3}z^3 \prec h(z).$$ These examples demonstrate that the class Mtan is rich in structure and includes a wide variety of analytic functions (see Figure [2\)](#page-6-0). <span id="page-6-0"></span> Figure 2. The figure illustrates the subordination of f ′ 0.38, g ′ , and i ′ with respect to h, shown in blue, green, and red, respectively. Building upon these approaches, this study introduces a novel image enhancement algorithm that convolves Hankel determinants derived from analytic functions associated with a cusp-shaped domain with image pixels using a 3 × 3 mask. This methodology extends existing computational techniques while addressing their limitations, particularly with respect to preserving structural integrity during enhancement.