On compact sets possessing $q$-convex functions

Proposition 2.5. Let $\Omega$ be an open set in $\mathcal{M}$ . Then the following properties are equivalent. - <span id="page-1-0"></span>1. $\Omega$ is q-pseudoconvex. - <span id="page-1-1"></span>2. For every spherical hat pair $(S, \widehat{S})$ of order n-q+1 such that $\Omega$ contains the spherical hat $S = S^{n-q+1}$ , the set $\Omega$ also contains the filled hat $\widehat{S} = \widehat{S}^{n-q+1}$ . Proof. (1) implies (2). We prove by contradiction. Assume that $\Omega$ fulfills (1) but not (2) of the above proposition. Then we can find a spherical hat $\mathbb{S}_r^{n-q+1}$ and an injective holomorphic map on a neighborhood U in $\mathbb{C}^n$ of $\widehat{\mathbb{S}}_r^{n-q+1} \times \Delta^{q-1}$ so that $\Phi(\mathbb{S}_r^{n-q+1} \times \Delta^{q-1})$ lies in $\Omega$ , but the complement of $\Omega$ intersects $\Phi(\operatorname{Int}(\widehat{\mathbb{S}}_r^{n-q+1} \times \Delta^{q-1}))$ . Define $D := \Phi^{-1}(\Omega)$ . Then D is q-pseudoconvex in U and, hence, admits the continuity principle with respect to (n-q)-dimensional analytic sets (cf., e.g., Theorem 4.3.2 in [8]). Let p be an intersection point of the interior of $\widehat{\mathbb{S}}_r^{n-q+1} \times \Delta^{q-1}$ and the boundary of D in U with coordinates $p = (p_1, p', p'')$ , where $p' = (p_2, \dots, p_{n-q+1})$ and $p'' = (p_{n-q+2}, \dots, p_n)$ . Now we define (n-q)-dimensional analytic sets as follows: $$A_t := \{(t+i\operatorname{Im}(p_1), z') \in \mathbb{C} \times \mathbb{C}^{n-q} : t^2 + (\operatorname{Im}(p_1))^2 + ||z'||^2 < 1\} \times \{p''\},\$$ with $t_0 := \text{Re}(p_1) \le t < \sqrt{1 - (\text{Im}(p_1))^2} =: t_1 \text{ and } z' = (z_2, \dots, z_{n-q+1})$ . Observe that - $\partial A_t := \{(t + i\operatorname{Im}(p_1), z') \in \mathbb{C} \times \mathbb{C}^{n-q} : t^2 + (\operatorname{Im}(p_1))^2 + ||z'||^2 = 1\} \times \{p''\}$ lies in $\mathbb{S}_r^{n-q+1} \times \Delta^{q-1} \subset D$ for every $t \in [t_0, t_1]$ , - $A_{t} \subset D$ for all $t^ < t_1$ close enough to $t_1$ , since $\partial A_{t_1} = \{(t_1 + i\operatorname{Im}(p_1), p', p'')\} \subset D$ , - and $p \in A_{t_0} \cap \partial D \neq \emptyset$ . Then we can find $t^{} \in (t_0, t^*)$ such that $A_{t^{}}$ intersects the boundary of D for the first time as $t \downarrow t_0$ . But this contradicts the continuity principle with respect to (n-q)-dimensional analytic sets of the q-pseudoconvex domain D in U. Hence, $\Omega$ has to fulfill property (2). (2) implies (1). We prove again by contradiction. Assume that $\Omega$ admits property (2), but is not q-pseudoconvex. Then there is a (q, n-q)-Hartogs figure $H_{r,s}^{q,n-q}$ and an injective holomorphic map $\Phi: \Delta^n \to \mathcal{M}$ such that $\Phi(H_{r,s}^{q,n-q}) \subset \Omega$ , but $\Phi(\Delta^n)$ intersects the complement of $\Omega$ . Let $D := \Phi^{-1}(\Omega)$ . Then, by assumption made on the Hartogs figure, $H_{r,s}^{q,n-q} \subset D$ , but $(H_{r,s}^{q,n-q})^c \cap D^c \neq \emptyset$ in $\Delta^n$ . Now let $$r_0 := \min\{r > 0 : H_{r,s}^{q,n-q} \cap D^c \neq \emptyset\}.$$ This means that the closure of $H^{q,n-q}_{r_0,s}$ intersects the boundary of D for the first time when the radius r increases towards 1. This intersection takes place at the boundary part of $H^{q,n-q}_{r_0,s}$ in $\Delta^n$ which lies in $\partial(\Delta^q_r) \times \Delta^{n-q}_s$ . Let p be such an intersecting point. Then there is an index $j_1 \in \{1,\ldots,q\}$ such that $$p \in \{p^\} \times \partial(\Delta_{r_0}) \times \{p^{*}\} \times \Delta_s^{n-q}$$ with $p^ := (p_1, \dots, p_{j_1-1}) \in \overline{\Delta_{r_0}^{j_1-1}}$ and $p^{} := (p_{j_1+1}, \dots, p_q) \in \overline{\Delta_{r_0}^{q-j_1-1}}$ . Consider the set $\Pi := \{p^\} \times \mathbb{C} \times \{p^{}\} \times \Delta^{n-q}$ and let $D' \subset \Delta^{n-q+1}$ be a domain such that $$\pi\Big(D\cap\Pi\Big)=D',$$ where $\pi: \mathbb{C}^n \to \mathbb{C}^{n-q+1}$ denotes the coordinate projection of the $(z_1, \ldots, z_n)$ -coordinates to the $(z_{j_1}, z'') = (z_{j_1}, z_{q+1}, \ldots, z_n)$ -coordinates. Observe that $$\pi(H_{r_0,s}^{q,n-q} \cap \Pi) = H' = \{|z_{j_1}| < r_0\} \cup \{||z''||_{\infty} > s\}$$ is an (1, n-q)-Hartogs figure in $\mathbb{C}^{n-q+1}$ such that $$H' \subset D'$$ , but $\Delta^{n-q+1} \cap (D')^c \neq \emptyset$ . Now we can use the same technique as in the second part of the proof of Proposition 2.1 in [11] and construct a spherical hat pair $(S', \hat{S}')$ of order n - q + 1 in $\mathbb{C}^{n-q+1}$ such that $$S' \subset D'$$ , but $\hat{S}' \cap (D')^c \neq \emptyset$ . We re-arrange the coordinates such that $z_{j_1}$ becomes $z_1$ , and z'' becomes $(z_2, \ldots, z_{n-q+1})$ . Then $$(S' \times \{(p^, p^{*})\}) \subset D$$ , but $(\hat{S}' \times \{(p^, p^{*})\}) \cap D^c \neq \emptyset$ . Recall that $(p^, p^{}) \in \Delta^{q-1}$ and notice that after a small pertubation, if necessary, we can assume that $(S' \times \{(p^, p^{})\}) \in D$ . Hence, there exists a small $\varepsilon \in (0, r)$ such that $$(S' \times \Delta_{\varepsilon}^{q-1}) \subset D$$ , but $(\hat{S}' \times \Delta_{\varepsilon}^{q-1}) \cap D^c \neq \emptyset$ . Let S := S ′ × ∆ q−1 ε and Sˆ := Sˆ′ × ∆ q−1 ε . Then we obtain a spherical hat pair (Φ(S), Φ(Sˆ)) of order n − q + 1 with Φ(S) ⊂ Ω, but Φ(Sˆ) ∩ Ω ^c ̸= ∅. This contradicts the assumption made on Ω to fulfill property [\(2\)](#page-1-1) of this proposition. Hence, Ω must be q-pseudoconvex. In what follows, we study the relation of q-pseudoconvex sets and q-convex functions. □

Proposition 2.11. Assume that an open set $\Omega$ in $\mathcal{M}$ admits an (n-q)-convex exhaustion function. Then $\Omega$ is q-pseudoconvex.

Lemma 3.3. For a finite family of sets $\{K_j''\}_{j=1}^{\ell}$ from $\mathcal{F}_K^q$ , the intersection $\bigcap_{j=1}^{\ell} K_j''$ also lies in $\mathcal{F}_K^q$ .

Proposition 3.4. The q-nucleus of $K \subseteq \mathcal{M}$ is q-pseudoconcave in $\mathcal{M}$ . Moreover, it is the maximal q-pseudoconcave subset of K. In particular, if K is q-pseudoconcave itself, then $\mathfrak{n}_q(K) = K$ .

Corollary 3.5. Let $\mathcal{P}_q(K)$ denote the collection of all q-pseudoconcave subsets of $K \subsetneq \mathcal{M}$ and $\mathcal{P}_q^*(K)$ the collection of all open q-pseudoconvex sets U in $\mathcal{M}$ such that $K \cap U = \emptyset$ . Then $$\mathfrak{n}_q(K) = \bigcup_{A \in \mathcal{P}_q(K)} A,$$ or, equivalently, $$\mathcal{M} \setminus \mathfrak{n}_q(K) = \bigcap_{U \in \mathcal{P}_q^*(K)} U.$$ In particular, $\mathcal{M} \setminus \mathfrak{n}_q(K)$ is q-pseudoconvex in $\mathcal{M}$ . The q-nucleus is a biholomorphic invariant in the following sense. Remark 3.6 Let K be a compact set in $\mathcal{M}$ . For a given biholomorphic map $\Phi : \mathcal{M} \to \mathcal{N}$ , where $\mathcal{N}$ is another complex manifold, and for a compact set K in $\mathcal{M}$ , we have that $$\Phi(\mathfrak{n}_q(K)) = \mathfrak{n}_q(\Phi(K)).$$ This immediately follows from Corollary 3.5 and from the fact that an open set U in $\mathcal{M}$ is q-pseudoconvex if and only $\Phi(U)$ is q-pseudoconvex in $\Phi(\mathcal{M}) = \mathcal{N}$ . We give an example of the q-nucleus in the projective manifold. Example 3.7 Consider the embedded $\mathbb{CP}^q$ in $\mathbb{CP}^n$ , $$\mathbb{CP}^q = \{ [z_0, \dots, z_n] \in \mathbb{CP}^n : z_{q+1} = \dots = z_n = 0 \}.$$ By computing the eigenvalues of $$\rho([z_0, \dots, z_n]) = \frac{|z_0|^2 + \dots + |z_q|^2}{|z_{q+1}|^2 + \dots + |z_n|^2}$$ using local charts, we find that $\rho$ is an (n-q)-convex exhaustion function for $\mathbb{CP}^n \setminus \mathbb{CP}^q$ . By Proposition 2.11, the set $\mathbb{CP}^q$ is a q-pseudoconcave compact subset of $\mathbb{CP}^n$ . Therefore, we can conclude that one has $\mathfrak{n}_q(\mathbb{CP}^q) = \mathbb{CP}^q$ in $\mathbb{CP}^n$ . Together with Corollary 2.12 we obtain the following result. It states that the manifold cannot be too nice in the geometric and holomorphic sense and that K has to be large enough in order to produce a proper q-nucleus. Corollary 3.8 The following two properties hold true: - 1. If $\mathcal{M}$ is q-complete (such as Stein in the case q=1), then $\mathfrak{n}_q(K)$ is empty for any compact set K in $\mathcal{M}$ . - 2. If K is contained in a local holomorphic chart of $\mathcal{M}$ , then $\mathfrak{n}_q(K) = \emptyset$ . Now we deal with the problem how to construct a q-convex function with corners in the neighborhood of K in the case when the q-nucleus of K is empty. First, we need the following lemma.

Lemma 3.9. Let K be a compact set in $\mathcal{M}$ and $V_1, V_2$ a pair of open sets in $\mathcal{M}$ such that K is contained in their union $V_1 \cup V_2$ . Assume that there are two (weakly) q-convex functions with corners $\varphi_j$ defined on some neighbourhoods of $\overline{V_j}$ for j=1,2 such that $\varphi_1 > \varphi_2$ on $\partial V_2 \cap V_1 \cap K$ and $\varphi_1 < \varphi_2$ on $\partial V_1 \cap V_2 \cap K$ . Define $$\Psi := \left\{ \begin{array}{l} \varphi_1 \ on \ V_1 \setminus V_2 \\ \varphi_2 \ on \ V_2 \setminus V_1 \\ \max\{\varphi_1, \varphi_2\} \ on \ V_1 \cap V_2 \end{array} \right.$$ Then there is a neighborhood W of K in $V_1 \cup V_2$ such that $\Psi$ is (weakly) q-convex with corners on W. Proof. It is obvious that $\max\{\varphi_1, \varphi_2\}$ is (weakly) q-convex with corners on $V_1 \cap V_2$ simply by definition. We show that it extends according to the definition of $\Psi$ outside $K \cap V_1 \cap V_2$ . For this, consider the compact sets $M_1 := \partial V_2 \cap V_1 \cap K$ and $M_2 := \partial V_1 \cap V_2 \cap K$ . By the assumptions made on $\varphi_1$ and $\varphi_2$ , there is an open neighborhood $U_1$ of $M_1$ in $V_1 \cup V_2$ such that $\varphi_1 > \varphi_2$ on $U_1 \cap V_1$ . Thus, $\max\{\varphi_1, \varphi_2\} = \varphi_1$ on $U_1 \cap V_1$ and easily extends by $\varphi_1$ into some open neighborhood $W_1$ of $K \setminus V_2$ to a q-convex function with corners. By the same arguments, we can extend $\max\{\varphi_1, \varphi_2\}$ by $\varphi_2$ into some open neighborhood $W_2$ of $K \setminus V_1$ . Now denote by $\Psi$ the above constructed extension of $\max\{\varphi_1, \varphi_2\}$ into the open neighborhood $W = W_1 \cup W_2 \cup (V_1 \cap V_2)$ of K.

Proposition 3.10. Let $(S, \widehat{S})$ be a spherical hat pair of order n - q + 1 in $\mathcal{M}$ . Then there is a neighborhood U of the filled hat $\widehat{S}$ and a weakly q-convex function $\varphi$ on $U \setminus S$ such that $\varphi$ vanishes on $U \setminus \widehat{S}$ and $\varphi$ is positive and q-convex on the filling $\operatorname{Int}(\widehat{S})$ .

Lemma 4.1. Given two compact sets K, L in $\mathcal{M}$ with $K \subset L$ , we have $\mathfrak{n}_q(K) \subset \mathfrak{n}_q(L)$ .

Proposition 4.3. Let A be a closed set in $\mathcal{M}$ . If there exists a q-convex function with corners in an open neighborhood U on A, then $\mathfrak{n}_q(A)$ is empty.

Proposition 4.5. The q-nucleus of a closed set $A \subseteq \mathcal{M}$ is q-pseudoconcave in $\mathcal{M}$ .

Definition 2.1. Fix integer numbers $k, m \ge 1$ and let 0 < r, s < 1. An Euclidean (k, m)-Hartogs figure is a set of the form $$H^{k,m} = H^{k,m}_{r,s} := \{ z \in \Delta^k \times \Delta^m : \|(z_1, \dots, z_k)\|_{\infty} < r \text{ or } \|(z_{k+1}, \dots, z_{k+m})\|_{\infty} > s \}.$$ 2010 Mathematics Subject Classification. Primary 32U05, 32F10; Secondary 32Q99. In the following, $\mathcal{M}^n = \mathcal{M}$ always denotes a complex manifold of dimension $n \geq 2$ unless otherwise stated.

Definition 2.2. Let $q \in \{1, \ldots, n-1\}$ be an integer number. An open set $\Omega$ in $\mathcal{M}$ is q-pseudoconvex (in $\mathcal{M}$ ) if it admits the Kontinuitätssatz with respect to (n-q)-polydiscs, i.e. for every (q, n-q)-Hartogs figure $H^{q,n-q}$ and every injective holomorphic map $\Phi$ : $\Delta^n \to \mathcal{M}$ such that $\Phi(H^{q,n-q}) \subset \Omega$ we have $\Phi(\Delta^n) \subset \Omega$ . Remark 2.3 Notice that the above described q-pseudoconvexity is defined in the sense of Rothstein [10]. It is equivalent to the (n-q-1)-pseudoconvexity in the sense of Słodkowski [12] (see [8] for a list of equivalent notions of q-pseudoconvexity). In the case q = n - 1, the (n-1)-pseudoconvexity in the sense of Rothstein is simply the classical pseudoconvexity. We introduce another notion of generalized pseudoconvexity which is based on half spheres rather than Hartogs figures.

Definition 2.4. Let 0 < r < 1 be a real number and fix some integer number $1 \le k \le n$ . We consider the spherical hat defined as $$\mathbb{S}_r^k := \{ z = (z_1, \dots, z_k) \in \mathbb{C}^k : ||z|| = 1 \text{ and } \operatorname{Re}(z_1) \ge r \}$$ and the filled (spherical) hat $$\widehat{\mathbb{S}}_r^k := \{ z = (z_1, \dots, z_k) \in \mathbb{C}^k : ||z|| \le 1 \text{ and } \operatorname{Re}(z_1) \ge r \}.$$ A pair of sets $(S^k, \widehat{S}^k) = (S, \widehat{S})$ in $\mathcal{M}$ is said to be a spherical hat pair of order k if there exist a real number 0 < r < 1, a filled spherical hat $\widehat{\mathbb{S}}_r^k$ , an open neighborhood U of $\widehat{\mathbb{S}}_r^k \times \overline{\Delta^{n-k}}$ in $\mathbb{C}^n$ and an injective holomorphic map $\Phi: U \to \mathcal{M}$ such that $$S = S^k := \Phi(\mathbb{S}^k_r \times \Delta^{n-k}) \quad \text{and} \quad \widehat{S} = \widehat{S}^k := \Phi\big(\widehat{\mathbb{S}}^k_r \times \Delta^{n-k}\big).$$ We also denote by $\operatorname{Int}(\widehat{S}^k)$ the image of the interior of $\widehat{\mathbb{S}}_r^k \times \Delta^{n-k}$ by $\Phi$ and call it the filling of $S^k$ . Now we compare the two notions of q-pseudoconvexity defined above.

Definition 3.1. Fix an integer number $q \in \{1, ..., n-1\}$ . - 1. Let $K'' \subset K' \subsetneq \mathcal{M}$ be two proper compact sets in $\mathcal{M}$ . We say that K'' is obtained from K' by a spherical cut of order k if there exists a spherical hat pair $(S, \widehat{S})$ of order k such that - $S \subset \mathcal{M} \setminus K'$ , - and $K'' = K' \setminus \operatorname{Int}(\widehat{S})$ . - 2. For two compacts $K'' \subset K'$ we say that K'' is obtained from K' by a sequence of spherical cuts of order k, if there exists a finite sequence $\{K_j\}_{j=1}^m$ of compact sets such that - $K_1 \supset K_2 \supset \cdots \supset K_m$ , - $K_1 = K'$ and $K'' = K_m$ , - and $K_{j+1}$ is obtained from $K_j$ by a spherical cut of order k for each $j = 1, \ldots, m-1$ . - 3. For a proper compact set $K \subsetneq \mathcal{M}$ we define $\mathcal{F}_K^q$ to be the set of all compacts K'' which are obtained from K by a sequence of spherical cuts of order n-q+1. Then we define the q-nucleus $\mathfrak{n}_q(K)$ of K as the intersection of all compacts $K'' \in \mathcal{F}_K^q$ , i.e., $$\mathfrak{n}_q(K) := \bigcap_{K'' \in \mathcal{F}_K^q} K''.$$ 4. Otherwise, if $K = \mathcal{M}$ , i.e., if $\mathcal{M}$ is compact itself, we simply set $\mathfrak{n}_a(\mathcal{M}) := \mathcal{M}$ . Remark 3.2 Notice that the q-nucleus is a compact set, and that the 1-nucleus corresponds to the nucleus introduced in [11]. We need the following lemma to derive some important properties of the q-nucleus.

Definition 4.2. Let $\{K_n\}_{n\in\mathbb{N}}$ be an arbitrary sequence of compact sets $\{K_n\}_{n\in\mathbb{N}}$ such that $K_k \subset K_{k+1}$ for every $n \in \mathbb{N}$ and $\mathcal{M} = \bigcup_k K_k$ . Then we define the q-nucleus for a closed set A in $\mathcal{M}$ by $$\mathfrak{n}_q(A) := \overline{\bigcup_{k \in \mathbb{N}} \mathfrak{n}_q(A \cap K_k)}.$$ We give a necessary condition on the existence of q-convex functions near closed sets.