Volume 23, Number 4, Fall/Winter 2023, Pages 509–682 Covered by: Mathematical Reviews zbMATH (formerly Zentralblatt MATH) COBISS SCOPUS Science Citation Index-Expanded (SCIE) Web of Science ISI Alerting Service Current Contents/Physical, Chemical & Earth Sciences (CC/PC & ES) dblp computer science bibliography The University of Primorska The Society of Mathematicians, Physicists and Astronomers of Slovenia The Institute of Mathematics, Physics and Mechanics The Slovenian Discrete and Applied Mathematics Society The publication is partially supported by the Slovenian Research Agency from the Call for co-financing of scientific periodical publications. Contents On girth-biregular graphs György Kiss, Štefko Miklavič, Tamás Szőnyi . . . . . . . . . . . . . . . . 509 An extension of the Erdős-Ko-Rado theorem to uniform set partitions Karen Meagher, Mahsa N. Shirazi, Brett Stevens . . . . . . . . . . . . . . 531 Almost simple groups as flag-transitive automorphism groups of symmetric designs with λ prime Seyed Hassan Alavi, Ashraf Daneshkhah, Fatemeh Mouseli . . . . . . . . . 553 A classification of connected cubic vertex-transitive bi-Cayley graphs over semidihedral group Jianji Cao, Young Soo Kwon, Mimi Zhang . . . . . . . . . . . . . . . . . . 563 Domination and independence numbers of large 2-crossing-critical graphs Vesna Iršič, Maruša Lekše, Mihael Pačnik, Petra Podlogar, Martin Praček . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577 Enumerating symmetric pyramids in Motzkin paths Rigoberto Flórez, José L. Ramírez . . . . . . . . . . . . . . . . . . . . . . 591 The core of a vertex-transitive complementary prism Marko Orel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605 Normal Cayley digraphs of dihedral groups with the CI-property Jin-Hua Xie, Yan-Quan Feng, Jin-Xin Zhou . . . . . . . . . . . . . . . . . 615 On cubic bi-Cayley graphs of p-groups Na Li, Young Soo Kwon, Jin-Xin Zhou . . . . . . . . . . . . . . . . . . . 631 Using a q-shuffle algebra to describe the basic module V (Λ0) for the quantized enveloping algebra Uq(ŝl2) Paul Terwilliger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 649 Volume 23, Number 4, Fall/Winter 2023, Pages 509–682 xxxi ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.01 / 509–530 https://doi.org/10.26493/1855-3974.2935.a7b (Also available at http://amc-journal.eu) On girth-biregular graphs György Kiss * Department of Geometry and ELKH-ELTE Geometric and Algebraic Combinatorics Research Group, Eötvös Loránd University, 1117 Budapest, Pázmány s. 1/c, Hungary and Faculty of Mathematics, Natural Sciences and Information Technologies, University of Primorska, Glagoljaška 8, 6000 Koper, Slovenia Štefko Miklavič † Andrej Marušič Institute, University of Primorska, Muzejski trg 2, 6000 Koper and Institute of Mathematics, Physics and Mechanics, Jadranska 19, 1000 Ljubljana, Slovenia Tamás Szőnyi ‡ Department of Computer Science and ELKH-ELTE Geometric and Algebraic Combinatorics Research Group, Eötvös Loránd University, 1117 Budapest, Pázmány s. 1/c, Hungary and Faculty of Mathematics, Natural Sciences and Information Technologies, University of Primorska, Glagoljaška 8, 6000 Koper, Slovenia Received 25 July 2022, accepted 12 November 2022, published online 13 February 2023 Abstract Let Γ denote a finite, connected, simple graph. For an edge e of Γ let n(e) denote the number of girth cycles containing e. For a vertex v of Γ let {e1, e2, . . . , ek} be the set of edges incident to v ordered such that n(e1) ≤ n(e2) ≤ · · · ≤ n(ek). Then (n(e1), n(e2), . . . , n(ek)) is called the signature of v. The graph Γ is said to be girth- biregular if it is bipartite, and all of its vertices belonging to the same bipartition have the same signature. *Corresponding author. This research was supported in part by the Hungarian National Research, Development and Innovation Office OTKA grant no. SNN 132625, by the HAS Slovenian-Hungarian bilateral research project ”Graph Colourings and Finite Geometries” (NKM-95/2019000206), and by the Slovenian Research Agency (re- search project J1-9110). †This research was supported in part by the Slovenian Research Agency (research program P1-0285 and research projects N1-0032, N1-0038, J1-5433, and J1-6720). ‡This research was supported in part by the Hungarian National Research, Development and Innovation Office OTKA grant no. K 124950, by the HAS Slovenian-Hungarian bilateral research project ”Graph Colourings and Finite Geometries” (NKM-95/2019000206), and by the Slovenian Research Agency (research project J1-9110). The author would like to thank his coauthors Gabriela, Alejandra and Robert for the paper published in 2022. The discussions leading to mentioned paper considerably influenced the present paper. The author author would like to thank the Rényi Institute of Mathematics for the hospitality during this research. cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 510 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 Let Γ be a girth-biregular graph with girth g = 2d and signatures (a1, a2, . . . , ak1) and (b1, b2, . . . , bk2), and assume without loss of generality that k1 ≥ k2. Our first result is that {a1, a2, . . . , ak1} = {b1, b2, . . . , bk2}. Our next result is the upper bound ak1 ≤ M , where M = (k1 − 1)⌊g/4⌋(k2 − 1)⌈g/4⌉. We describe the graphs attaining equality. For d = 3 or d ≥ 4 even they are incidence graphs of Steiner systems and generalized polygons, respectively. Finally, we show that when d is even and ak1 = M −ε for some non-negative integer ε < k2 − 1, then ε = 0. Similar result is valid for d = 3, ε ≤ 1 and k2 ̸ | k1. Keywords: Girth cycle, girth-biregular graph, Steiner system, generalized polygons. Math. Subj. Class. (2020): 05C35, 51E20 1 Introduction In extremal graph theory one often considers problems of the following type: we fix some graph parameter or some graph property and want to deduce the extremal number of another parameter (in many cases the number of points or edges). Typical questions are Turán type problems, see e.g. the survey of Füredi and Simonovits [7]. The problem considered in our paper is motivated by the cage problem (and the degree/diameter problem), see [4, 12]. The cage problem was extended recently by several authors to bipartite graphs which are biregular in the sense that vertices in the same bipartition set have the same degree, see Jajcay, Ramos-Rivera and their coauthors [1, 6]. The paper by Jajcay, Kiss and Miklavič [8] defined a new type of regularity: a graph is called edge-girth regular if the number of cycles of length g (the girth) containing an edge is independent of the edge. This definition was weakened by Potočnik and Vidali [14] and in [9] it was extended to a stability theorem. One can introduce the signature (a1, . . . ak) of a point as the ordered sequence of the number of girth cycles containing the edges emanating from the point (see Definition 2.1). A graph is called girth-regular if all of its points have the same signature. For such graphs with valency k ≥ 3, it was shown in [14] that ak ≤ (k − 1)2d, where d = ⌊g/2⌋. In [9], the upper bound was improved for g = 2d in the sense that it is either (k − 1)2d or at most (k − 1)2d − (k − 1). In the former case the graph has to be the incidence graph of a thick generalized d-gon of order (k − 1, k − 1). In particular, we must have d = 2, 3, 4, 6. The aim of the present paper is to extend some of the results of [9] to the bipartite biregular case. If the valencies in the bipartition classes are k1 > k2 > 2, then we prove that the maximum number of girth-cycles containg an edge is at most M = (k1 − 1)⌊g/4⌋(k2 − 1)⌈g/4⌉, see Theorem 2.6. For g = 4, we show that when the graph is girth regular and the largest element of the signature of a point is equal to M − ε, with ε ≤ k2 − 1, then ε = 0, and the graph is the complete bipartite graph Kk1,k2 . In Section 3, we prove an analogous result for g = 2d ≥ 8, d even, relating the ε = 0 case to incidence graphs of a finite thick generalized d-gon, see Theorem 3.4(vi). For q = 2d, d odd, we have partial results. In particular, similarly to the results of [1, 6], when g = 6, we could find a connection of ak = M and block designs. For particular k1 and k2, the connection is with affine planes, see Corollary 6.3. E-mail addresses: gyorgy.kiss@ttk.elte.hu (György Kiss), stefko.miklavic@upr.si (Štefko Miklavič), tamas.szonyi@ttk.elte.hu (Tamás Szőnyi) Gy. Kiss et al.: On girth-biregular graphs 511 2 Definitions and basic properties In this section we collect basic notation and terminology. First, for the sake of complete- ness, we recall some definitions from design theory and finite geometries. In the second subsection we define girth-biregular graphs and present some simple, important properties of them. 2.1 Block designs, Steiner systems, generalized polygons Here we give only the most necessary definitions. A detailed introduction to block de- signs and Steiner systems we refer the reader to [2] and [3], while the concepts from finite geometries we use can be found for example in [10] and [11]. Definition 2.1. Let v ≥ k ≥ t ≥ 2 and λ ≥ 1 be integers. A t-(v, k, λ) design is a collection of k-subsets (blocks) of a v-set S (points) such that every t-subset of S is contained in exactly λ of the blocks. A t-(v, k, 1) design is called a Steiner system. In particular, the blocks of a Steiner system with t = 2 are often called lines. A parallelism of a design is a partition of its blocks into classes C1, C2, . . . , Cr with the property that any point belongs to a unique block of each class. A design is called resolvable, if it has a parallelism. Let (P,L, I) be a connected, finite point-line incidence geometry. The elements of P and L are called points and lines, respectively, I ⊂ (P×L)∪(L×P) is a symmetric relation, called incidence. A chain of length h is a sequence x0 Ix1 I . . . Ixh where xi ∈ P ∪ L. The distance of the elements u and v, denoted by d(u, v), is the length of the shortest chain joining them. Definition 2.2. Let n > 1 be a positive integer. The incidence geometry G = (P,L, I) is called a thick generalized n-gon if it satisfies the following axioms. • d(x, y) ≤ n ∀ x, y ∈ P ∪ L. • If d(x, y) = k < n, then there is a unique chain of length k joining x and y. • ∀ x ∈ P ∪ L ∃ y ∈ P ∪ L such that d(x, y) = n. • ∀ x ∈ P ∪ L there exist at least three elements yi ∈ P ∪ L such that d(x, yi) = 1. For any finite thick generalized n-gon G there exist integers s, t ≥ 2 such that every line is incident with exactly s+1 points and every point is incident with exactly t+1 lines. The pair (s, t) is called the order of G. In particular, for n = 3, the generalized 3-gons are the finite projective planes, for n = 4, the generalized 4-gons are the finite generalized quadrangles (GQ-s for short). The GQ-s have an alternative definition: Definition 2.3. Let s > 1 and t > 1 be positive integers. A thick generalized quadrangle of order (s, t) is a point-line incidence structure which satisfies the following axioms: • every line is incident with exactly s+ 1 points; • every point is incident with exactly t+ 1 lines; 512 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 • there exists a non-incident point-line pair; • for every point P and every line ℓ not incident with P , there is exactly one line through P which intersects ℓ. 2.2 Girth-biregular graphs Let Γ denote a finite, connected, simple graph with vertex set V = V (Γ) and edge set E = E(Γ). Let d denote the minimal path-length distance function of Γ and let D = max{d(v, w) | v, w ∈ V } denote the diameter of Γ. For v ∈ V and an integer i we let Γi(v) = {w ∈ V | d(v, w) = i}. We abbreviate Γ(v) = Γ1(v) and observe that Γi(v) = ∅ whenever i < 0 or i > D. For an edge uv of Γ, let Dij(u, v) = Γi(u) ∩ Γj(v). We say that Γ is biregular with valencies k1, k2 (k ∈ Z), whenever Γ is bipartite with bipartition sets A,B, and |Γ(v)| = k1 (|Γ(v)| = k2, respectively) for every v ∈ A (v ∈ B, respectively). If Γ is not a tree, then the girth g of Γ is the length of a shortest cycle in Γ. If C is a cycle of Γ of girth length g, then we refer to C as a girth cycle of Γ. The incidence graph (also known as Levi graph) of a point-line incidence geometry is a bipartite graph whose bipartition sets correspond to the set of points and lines, respectively, and there is an edge between two vertices if and only if the corresponding point is incident with the corresponding line. The next ”folklore” statement gives an important correspondence between generalized polygons and biregular graphs. The proof can be found for example in [11, Lemma 1.3.6], or in [10, Chapter 12]. Theorem 2.4. A finite thick generalized n-gon G exists if and only if there exists a con- nected bipartite biregular graph Γ of diameter n and girth 2n, such that each vertex has degree at least three. In this case Γ is the incidence graph of G. The following definition is a central definition of this paper. Definition 2.5. Let Γ be a graph and let u, v be adjacent vertices of Γ. For the edge e = uv of Γ let n(e) = n(uv) denote the number of girth cycles containing e. For a vertex w of Γ let {e1, e2, . . . , ek(w)} be the set of edges incident to w ordered such that n(e1) ≤ n(e2) ≤ · · · ≤ n(ek(w)). Then (n(e1), n(e2), . . . , n(ekw)) is called the signature of w. The bipartite graph G is said to be girth-biregular if all of its vertices belonging to the same bipartition have the same signature. Observe that girth-biregular graphs are also biregular. The following straightforward observation will be used through the rest of the paper frequently without explicitly referring to it (see also [14, Subsection 2.2] and Figure 1). Proposition 2.6. Let Γ be a biregular graph with valencies k1, k2 and girth 2d, d ≥ 2. Let uv be an edge of Γ, such that the valency of u is k1 and valency of v is k2. Let Dij = D i j(u, v). Then the following hold. (i) If x, y are vertices of Γ with d(x, y) ≤ d − 1, then there is a unique path of length d(x, y) between x and y. (ii) Dii = ∅ for every integer i. (iii) For 1 ≤ i ≤ d−1 and for z ∈ Dii+1 (resp. z ∈ D i+1 i ), we have that |Γ(z)∩D i−1 i | = 1 (resp. |Γ(z) ∩Dii−1| = 1) . Gy. Kiss et al.: On girth-biregular graphs 513 (iv) For 0 ≤ i ≤ d − 2 and for z ∈ Dii+1, we have that |Γ(z) ∩D i+1 i+2| = k1 − 1 if i is even, and |Γ(z) ∩Di+1i+2| = k2 − 1 if i is odd. (v) For 0 ≤ i ≤ d − 2 and for z ∈ Di+1i , we have that |Γ(z) ∩D i+2 i+1| = k2 − 1 if i is even, and |Γ(z) ∩Di+1i+2| = k1 − 1 if i is odd. (vi) For 0 ≤ i ≤ d− 1 we have that |Dii+1| = { (k1 − 1)i/2(k2 − 1)i/2 if i is even, (k1 − 1)(i+1)/2(k2 − 1)(i−1)/2 if i is odd. |Di+1i | = { (k1 − 1)i/2(k2 − 1)i/2 if i is even, (k1 − 1)(i−1)/2(k2 − 1)(i+1)/2 if i is odd. (vii) There are exactly n(uv) edges between Dd−1d and D d d−1. 1 1 11 edges n(uv) Ddd+1 Ddd-1 Dd+1dv u 1 1 11 k -11 2 1 2 k -1 D12 D12 D23 D23 Dd-2d-1 Dd-1d-2 Dd-1dk-1 k -1 1 k -1 k -12 k -11 k -12 Figure 1: A biregular graph with valencies k1, k2 and girth 2d, d odd. The numbers near the bubble representing the set Dij represent the number of neighbours that each vertex of Dij has in the neighbouring bubble. 3 Some properties of girth-biregular graphs In this section we continue to study girth-biregular graphs. We prove several results about these graphs that are interesting on their own, and that will also be useful in the rest of the paper. Keeping in mind Proposition 2.6, one can calculate the number of girth cycles containing two fixed edges. Lemma 3.1. Let Γ be a girth-biregular graph with valencies k1 ≥ k2 and girth g = 2d. Let u1u2 and v1v2 be two edges of Γ. Without loss of generality we may assume that d(u1, v1) = min{d(ui, vj) : 1 ≤ i, j ≤ 2}. Let m = d(u1, v1) + 1, and let c denote the number of girth cycles containing both u1u2 and v1v2. Then c = 0 if m ≥ d+1 and c ≤ 1 if m = d. Moreover, if m ≤ d− 1, then c ≤  (k1 − 1)(d−m)/2(k2 − 1)(d−m)/2, if m and d are of the same parity, (k1 − 1)(d−1−m)/2(k2 − 1)(d+1−m)/2, if m is even and d is odd, (k1 − 1)(d−1−m)/2(k2 − 1)(d+1−m)/2, if m is odd, d is even and valency of v2 is k2, (k1 − 1)(d+1−m)/2(k2 − 1)(d−1−m)/2, if m is odd, d is even and valency of v2 is k1. 514 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 Proof. The statement is obvious if m ≥ d + 1. If m = d, then d − 1 = d(u1, v1) ≤ d(u2, v2), so there exists a girth cycle containing both u1u2 and v1v2 if and only if d(u2, v2) = d− 1, hence c ≤ 1. Suppose that m ≤ d − 1. Let Dij = Dij(u1, u2) and observe that v1 ∈ Dm−1m , v2 ∈ Dmm+1. Note that there is a unique path of length m − 1 between v1 and u1. Let F = Dd−m−1d−m (v2, v1) and note that by Proposition 2.6(iii) we have that F ⊆ D d−1 d . Let us denote the valency of v2 by k and let k′ be the other valency of Γ. Then |F | = (k − 1)⌈(d−m−1)/2⌉(k′ − 1)⌊(d−m−1)/2⌋, and there is a unique path of length d −m − 1 between v2 and any element of F because the girth of Γ is 2d. Now the number of girth cycles containing both u1u2 and v1v2 equals to the number of edges between F and Ddd−1. Observe that this number is the same as the number of (d −m)-arcs (v2, x1, . . . , f, r) where f ∈ F and r ∈ Dd−1d . Observe also that the valency of f is k if d −m − 1 is even and it is k′ if d −m − 1 is odd. Therefore, we have that c ≤ |F |(k − 1) if d −m − 1 is even, and c ≤ |F |(k′ − 1) if d −m − 1 is odd. Now we distinguish four cases. If d and m are of the same parity, then d −m − 1 is odd, and so c ≤ |F |(k′ − 1) = (k − 1)(d−m)/2(k′ − 1)(d−m)/2 = (k1 − 1)(d−m)/2(k2 − 1)(d−m)/2. If d is odd and m is even, then deg(u2) ̸= deg(v2), so we may assume deg(v2) = k2 (otherwise we interchange the roles of edges u1u2 and v1v2). Hence c ≤ |F |(k − 1) = |F |(k2 − 1) = (k1 − 1)(d−m−1)/2(k2 − 1)(d−m+1)/2. Finally, if d is even and m is odd, then c ≤ |F |(k − 1) = (k − 1)(d−m+1)/2(k′ − 1)(d−m−1)/2, and this gives the third and fourth estimates of the statement according as k = k1 or k = k2. Proposition 3.2. Let Γ be a girth-biregular graph with bipartition A,B and valencies k1 ≥ k2. Let us denote the signature of vertices from A by (a1, a2, . . . , ak1) and the signature of vertices from B by (b1, b2, . . . , bk2). Then {a1, a2, . . . , ak1} = {b1, b2, . . . , bk2}. Proof. As Γ is bipartite, each edge e of Γ is incident with one vertex from A and with one vertex from B. It thus follows that n(e) ∈ {a1, a2, . . . , ak1} if and only if n(e) ∈ {b1, b2, . . . , bk2}. This shows that {a1, a2, . . . , ak1} = {b1, b2, . . . , bk2}. Proposition 3.3. Let Γ be a girth-biregular graph with bipartition A,B and valencies k1 ≥ k2. Let us denote the signature of vertices from A by (a1, a2, . . . , ak1) and the signature of vertices from B by (b1, b2, . . . , bk2). Pick a ∈ {a1, a2, . . . , ak1} = {b1, b2, . . . , bk2}. Let aA (aB , respectively) denote the number of appearences of a in the signature (a1, a2, . . . , ak1) ((b1, b2, . . . , bk2), respectively). Then k2aA = k1aB . Proof. Let us count the number of edges of Γ that are contained in exactly a girth cycles. On the one hand, this number is equal to |A|aA, and on the other hand it is equal to |B|aB . Recall also that |A|k1 = |B|k2. The claim follows. Gy. Kiss et al.: On girth-biregular graphs 515 Let Γ be a girth-biregular graph with bipartition A,B and valencies k1 ≥ k2. Let us de- note the signature of vertices from A by (a1, a2, . . . , ak1) and signature of vertices from B by (b1, b2, . . . , bk2). Let us comment on the case k1 = k2. It follows from Proposition 3.3 that in this case we have aA = aB for every a ∈ {a1, a2, . . . , ak1} = {b1, b2, . . . , bk2}. Therefore, Γ is in fact girth-regular graph. As girth regular graphs were studied in details in [9] and [14], we will assume k1 > k2 for the rest of this paper. Observe also that connected biregular graphs with valencies k1, k2 = 1 are just the star graphs, which contain no cycles at all (and are therefore girth-biregular with signatures (0, 0, . . . , 0) and (0)). Let Γ be a girth-biregular graph with bipartition A,B and valencies k1 > k2 = 2. Then for any vertex w ∈ B there are two edges, say u1w and u2w through w, hence a cycle contains u1w if and only if it contains u2w. In particular, n(u1w) = n(u2w) which implies b1 = b2. Now, define the graph Γ′ in the following way: V (Γ′) = A and there is an edge between vertices u and v if and only if d(u, v) = 2 in Γ. Then Γ′ is an edge-girth- regular graph with valency k1. These graphs were studied in [8]. Therefore, in the rest of this paper we also assume k1 > k2 > 2. The following theorem is a generalization of the result of Potočnik and Vidali [14, Theorem 1.3]. Theorem 3.4. Let Γ be a girth-biregular graph with bipartition A,B, valencies k1 > k2 > 2 and girth 2d. Let us denote the signature of vertices from A by (a1, a2, . . . , ak1) and the signature of vertices from B by (b1, b2, . . . , bk2). Let M = (k1 − 1)g/4(k2 − 1)g/4 if d is even, and M = (k1 − 1)(g−2)/4(k2 − 1)(g+2)/4 if d is odd. Then ak1 = bk2 ≤ M . When the upper bound is attained, ak1 = bk2 = M, the following (i)-(vii) hold. (i) For every edge uv of Γ with u ∈ A and n(uv) = M we have Di+1i (u, v) = ∅ for i ≥ d. (ii) The signature of each vertex of Γ is (M,M, . . . ,M), hence n(e) = M for all e ∈ E(Γ). (iii) Every path on d + 2 vertices of Γ, starting in a vertex that is contained in A, is contained in a unique girth cycle; (iv) If d is even and uv is an edge of Γ with u ∈ A, then Dii+1(u, v) = ∅ for i ≥ d. (v) if d is odd and uv is an edge of Γ with u ∈ A, then Ddd+1(u, v) ̸= ∅ and Dii+1 = ∅ for i ≥ d+ 1. (vi) if d is even, then Γ is the incidence graph of a generalized d-gon of order (k1 − 1, k2 − 1); (vii) if d = 3, then Γ is the incidence graph of a 2− (k1k2 − k1 + 1, k2, 1)-design. Proof. Pick adjacent vertices u ∈ A, v ∈ B such that n(uv) = ak1 = bk2 . We prove the upper bound on ak1 in the case when d is odd. The proof for the case when d is even is similar. By Proposition 2.6(vi) we have that |Ddd−1(u, v)| = (k1 − 1)(d−1)/2(k2 − 1)(d−1)/2. As Ddd−1(u, v) ⊆ B and as every vertex from Ddd−1(u, v) has exactly one neighbour in Dd−1d−2(u, v), it follows that every vertex from Ddd−1(u, v) has at most k2 − 1 neighbours in D d−1 d (u, v). Therefore, there are at most 516 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 (k1 − 1)(d−1)/2(k2 − 1)(d+1)/2 edges between Ddd−1(u, v) and D d−1 d (u, v). The result now follows from Proposition 2.6(vii). Now, suppose that ak1 = M. (i): By Proposition 2.6(vii), there are M edges between Ddd−1(u, v) and D d−1 d (u, v). Re- call that by Proposition 2.6(iii), every vertex from Ddd−1(u, v) has exactly one neighbour in Dd−1d−2(u, v). It follows that every vertex from D d d−1(u, v) has all other neighbours in Dd−1d , and so D d+1 d (u, v) = ∅. Consequently, D i+1 i (u, v) = ∅ for every i ≥ d. (ii): Let w ∈ D21(u, v) be any vertex. Then we have that Dd−1d (v, w) = D d−2 d−1(u, v) ∪ ( Ddd−1(u, v) \Dd−2d−1(w, v) ) , and, as Dd+1d (u, v) = ∅ by (i) above, also Ddd−1(v, w) = D d−1 d (u, v). By Proposition 2.6(iv), the number of edges between Dd−2d−1(u, v) and D d−1 d (u, v) is equal to |Dd−2d−1(u, v)|(k2 − 1) if d is odd, and to |D d−2 d−1(u, v)|(k1 − 1) if d is even. As every vertex from Ddd−1(u, v) has exactly one neighbour in D d−1 d−2(u, v) and as D d+1 d (u, v) = ∅, the number of edges between ( Ddd−1(u, v) \D d−2 d−1(w, v) ) and Dd−1d (u, v) is equal to( |Ddd−1(u, v)| − |Dd−2d−1(w, v)| ) (k2 − 1) if d is odd, and to ( |Ddd−1(u, v)| − |Dd−2d−1(w, v)| ) (k1 − 1) if d is even. Observe that by Proposition 2.6(vi) we have that |Dd−2d−1(u, v)| = |D d−2 d−1(w, v)|, and so Proposition 2.6(vii) and the above comments imply that n(vw) = (k2 − 1)|Ddd−1(u, v)| if d is odd and n(vw) = (k1−1)|Ddd−1(u, v)| if d is even. Finally, Proposition 2.6(vi) implies that n(vw) = M . Hence the signature of v is (M,M, . . . ,M), so the girth-biregularity of Γ implies that n(e) = M for all e ∈ E(Γ). (iii): Pick any path x0x1 . . . xd+1 with x0 ∈ A and consider the sets Dij(x0, x1). It follows from Proposition 2.6 that xi ∈ Dii−1 for 1 ≤ i ≤ d. Recall that n(x0x1) = M by (ii) above, and so Dd+1d (x0, x1) = ∅ by (i) above. It follows that xd+1 ∈ D d−1 d . The result now follows from Proposition 2.6(iii). (iv): Recall that by (ii) above we have n(uv) = M , and so there are exactly M edges between Dd−1d (u, v) and D d d−1(u, v). Recall also that by Proposition 2.6(iii), every vertex from Dd−1d (u, v) has exactly one neighbour in D d−2 d−1(u, v). It follows that every vertex from Dd−1d (u, v) has all other neighbours in D d d−1, and so D d d+1(u, v) = ∅. Consequently, Dii+1(u, v) = ∅ for every i ≥ d. (v): By Proposition 2.6(vi) we have |Dd−1d (u, v)| = (k1−1)(d−1)/2(k2−1)(d−1)/2. As ver- tices of Dd−1d (u, v) have valency k1, there are therefore k1(k1 − 1)(d−1)/2 (k2 − 1)(d−1)/2 edges going out of Dd−1d (u, v). As n(uv) = M by (ii) above, M = (k1−1)(d−1)/2(k2−1)(d+1)/2 of these edges are between Dd−1d (u, v) and Ddd−1(u, v). By Proposition 2.6(iii), (k1−1)(d−1)/2(k2−1)(d−1)/2 of these edges are between Dd−1d (u, v) and Dd−2d−1(u, v). It follows that there are exactly (k1 − k2)(k1 − 1)(d−1)/2(k2 − 1)(d−1)/2 edges between Dd−1d (u, v) and D d d+1(u, v). As k1 > k2 ≥ 3, this number is nonzero, implying that Ddd+1(u, v) ̸= ∅. Gy. Kiss et al.: On girth-biregular graphs 517 Assume now that Dd+1d+2(u, v) ̸= ∅. Pick w ∈ D d+1 d+2(u, v) and let ux1x2 . . . xdw be arbitrary path between u and w such that xi ∈ Dii+1 for 1 ≤ i ≤ d. Note that this path is not contained in a girth cycle of Γ, contradicting (iii) above. Therefore Dd+1d+2(u, v) = ∅ and consequently Dii+1(u, v) = ∅ for every i ≥ d+ 1. (vi): Observe that (i), (ii) and (iv) above implies that the diameter of Γ is d. As k1 > k2 ≥ 3, Theorem 2.4 implies that Γ is the incidence graph of a generalized d-gon. (vii): Finally, suppose that d = 3. We call the vertices in A points and the the vertices in B lines and we use the geometric terminology. We claim that there is a unique line through any pair of distinct points. As the girth of Γ is 6, there is at most one line through any pair of points. Pick now distinct points x, y ∈ A. Pick an arbitrary line z through x. It follows from (i) and (v) above, that either y ∈ D21(x, z) or y ∈ D23(x, z). If y ∈ D21(x, z) , then z is the unique line through x and y. If however y ∈ D23(x, z), then, by Proposition 2.6(iii), there is a unique line w ∈ D12(x, z) which is adjacent to both x and y in Γ. Therefore, in this case w is the unique line through x and y. In the rest of this paper we use the following notation. Notation 3.5. Let Γ be a girth-biregular graph with bipartition A,B, valencies k1 > k2 ≥ 3, girth g = 2d, signatures (a1, a2, . . . , ak1) and (b1, b2, . . . , bk2). Let M = (k1 − 1)g/4(k2 − 1)g/4 if d is even, and M = (k1 − 1)(g−2)/4(k2 − 1)(g+2)/4 if d is odd and suppose that ak1 = M − ε for some ε < k2 − 1. Let uv be an edge with u ∈ A, v ∈ B and n(uv) = ak1 , and let D i j = D i j(u, v). Note that D i i = ∅ for every i and that there are no edges between Di−1i and D i i−1 for 1 ≤ i ≤ d− 1. For every r ∈ Ddd−1 (s ∈ D d−1 d , respectively) we let h(r) = |Γ(r) ∩ D d+1 d | (h(s) = |Γ(s)∩Ddd+1|, respectively). Let {r1, r2, . . . , rm} ⊆ Ddd−1 be the set of vertices of Ddd−1, for which the value of the function h is positive, that is, the set of vertices of Ddd−1, that have a neighbour in Dd+1d . Choose the indices in such a way that h(ri) ≤ h(rj) for i < j. Similarly, let {s1, s2, . . . , sn} ⊆ Dd−1d be the set of vertices of D d−1 d , for which the value of the function h is positive. Again, choose the indices in such a way that h(si) ≤ h(sj) for i < j. We also set γ = h(rm), σ = h(sn), µ = h(r1) and ν = h(s1). Proposition 3.6. Suppose that g = 2d with d even. With reference to Notation 3.5, we have∑ r∈Ddd−1 h(r) = m∑ i=1 h(ri) = ∑ s∈Dd−1d h(s) = n∑ i=1 h(si) = ε. (3.1) Proof. The first and the third of the above equalities are clear. We now prove that∑n i=1 h(si) = ε. The proof that ∑m i=1 h(ri) = ε is similar. Let E denote the set of edges, that have one endpoint in Dd−1d , and the other endpoint in D d d+1. Note that E = ∑n i=1 h(si), and so it is enough to prove |E| = ε. As d is even, it follows from Propo- sition 2.6(vi) that |Dd−1d | = (k1 − 1)d/2(k2 − 1)(d−2)/2. As D d−1 d ⊆ B, there are total (k1−1)d/2(k2−1)(d−2)/2k2 edges, having one endpoint in Dd−1d . By Proposition 2.6(iii), (k1 − 1)d/2(k2 − 1)(d−2)/2 of these edges have the other endpoint in Dd−2d−1 . Since ak = M − ε, it follows from Proposition 2.6(vii) that there are (k1 − 1)d/2(k2 − 1)d/2 − ε edges between Dd−1d and D d d−1. Combining these observations, we get the desired result. Lemma 3.7. Suppose that g = 2d with d even. With reference to Notation 3.5, we have m ≥ σ and n ≥ γ. 518 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 Proof. Set Γ(u) \ {v} = {u1, u2, . . . , uk1−1} and Γ(v) \ {u} = {v1, v2, . . . , vk2−1}. Moreover, for 1 ≤ i ≤ k1 − 1 (1 ≤ i ≤ k2 − 1, respectively) set Ui = Γd−2(ui) ∩Dd−1d (Vi = Γd−2(vi) ∩ Ddd−1, respectively). Note that as girth of Γ is 2d, the sets Ui (Vi, respectively) are pairwise disjoint, and |Ui| = |Vi| = (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2. Moreover, each r ∈ Ddd−1 (s ∈ D d−1 d , respectively) could have at most one neighbour in Ui (Vi, respectively) for each i. It is now clear that if s ∈ Dd−1d has no neighbours in Vi for some 1 ≤ i ≤ k2 − 1, then there is at least one vertex r ∈ Vi with h(r) ≥ 1. It follows m ≥ σ. Similarly we show that n ≥ γ. Equation (3.1) and Lemma 3.7 obviously imply the following inequalities: µσ ≤ µm ≤ ε, νγ ≤ νn ≤ ε. (3.2) If γ ≤ σ, then observe also that it follows from the above comments that µ2 ≤ µγ ≤ µσ ≤ µm ≤ ε, while if σ ≤ γ then ν2 ≤ νσ ≤ νγ ≤ νn ≤ ε. This shows that if γ ≤ σ then µ ≤ √ ε, while if σ ≤ γ then ν ≤ √ ε. First, we give a lower bound on a1 using the vertex u. Lemma 3.8. With reference to Notation 3.5 we have that a1 ≥ (k1−1)(d−2)/2(k2−1)(d−2)/2 max{(k2−1−σ)(k1−1), (k1−1−γ)(k2−1)}−ε. (3.3) Proof. We prove that a1 ≥ (k1 − 1)d/2(k2 − 1)(d−2)/2(k2 − 1 − σ) − ε. The proof of a1 ≥ (k1 − 1)(d−2)/2(k2 − 1)d/2(k1 − 1− γ)− ε is similar. Recall that n(uv) = ak and that Dij = D i j(u, v). Let s ̸= v be a neighbour of u such that n(us) = a1. Abbreviate K = Dd−1d ∩ Γd−2(s). For s′ ∈ K abbreviate L(s′) = Ddd−1 ∩ Γ(s′). Note that as girth of Γ is 2d, we have that sets L(s′) are pairwise disjoint, and so by (3.1) we have that ∑ s′∈K ∑ r′∈L(s′) h(r′) ≤ ε. Pick r′ ∈ L(s′) and observe that for each r̃ ∈ (Γ(r′) ∩ (Dd−1d ∪D d−1 d−2)) \ {s′}, there is a unique girth cycle containing the arc us and the 2-arc s′r′r̃. Note that Gy. Kiss et al.: On girth-biregular graphs 519 |K| = (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2, and so, by (3.1), we have a1 = n(us) ≥ ∑ s′∈K ∑ r′∈L(s′) (k1 − 1− h(r′)) = ∑ s′∈K ∑ r′∈L(s′) (k1 − 1)− ∑ s′∈K ∑ r′∈L(s′) h(r′) ≥ (k1 − 1) ∑ s′∈K (k2 − 1− h(s′))− ε ≥ (k1 − 1) ∑ s′∈K (k2 − 1− σ)− ε = (k1 − 1)(k1 − 1)(d−2)/2(k2 − 1)(d−2)/2(k2 − 1− σ)− ε = (k1 − 1)d/2(k2 − 1)(d−2)/2(k2 − 1− σ)− ε. 4 The case g = 4 In this section we consider the case g = 4. Throughout this section we will use Nota- tion 3.5. Recall that m (n, respectively) denotes the number of vertices of Ddd−1 (D d−1 d , respectively), for which the value of the function h is positive. Lemma 4.1. Assume that g = 4 and ε ≥ 1. Pick 1 ≤ i ≤ n, 1 ≤ j ≤ m, w ∈ Γ(si) ∩D23 and w̃ ∈ Γ(rj) ∩D32 . Then the following (i) – (iv) holds. (i) There are at most (h(si) − 1)(k1 − 1) girth cycles of the form (w, si, x, y, w) such that x ∈ Γ(si) ∩D23 . (ii) There are at most ε girth cycles of the form (w, si, x, y, w) such that x ∈ D21 and y ̸∈ D12 . (iii) There are at most (h(rj) − 1)(k2 − 1) girth cycles of the form (w̃, rj , x, y, w̃) such that x ∈ Γ(rj) ∩D32 . (iv) There are at most ε girth cycles of the form (w̃, rj , x, y, w̃) such that x ∈ D12 and y ̸∈ D21 . Proof. (i): Note that there are h(si) − 1 choices for x, and for each such choice there are at most k1 − 1 choices for y. The result follows. (ii): As x ∈ D21 and y ̸∈ D12 , it follows that y ∈ D32 . It follows from Proposition 3.6 that there are at most ε choices for the edge xy. For each such edge xy there is clearly at most one girth cycle containing also the edge wsi. The result follows. (iii), (iv): Similar to the proofs of (i) and (ii) above. Lemma 4.2. Assume that g = 4 and ε ≥ 1. Then m ≥ 2 and n ≥ 2. Proof. We prove that n ≥ 2. The proof that m ≥ 2 is similar. Suppose on the contrary that n = 1. Note that in this case σ = ν = ε, γ = 1, m = ε and h(ri) = 1 for 1 ≤ i ≤ m. Let w be the unique neighbour of r1 in D32 . Let t = |Γ(w) ∩ D21| and note that t ≤ m = ε. Note that the girth cycles containing the edge r1w are exactly the cycles of form (w, r1, x, y, w), where x ∈ {v} ∪ (D12 \ {s1}) and y ∈ (Γ(w) ∩ D21) \ {r1}. 520 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 Therefore, n(r1w) ≤ (k1 − 1)(t− 1) ≤ (k − 1)(ε− 1). Since γ = 1 and σ = ε, we have by Lemma 3.8 that a1 ≥ max{(k2 − 1− ε)(k1 − 1), (k1 − 2)(k2 − 1)} − ε ≥ (k1 − 2)(k2 − 1)− ε, and so (k1 − 2)(k2 − 1)− ε ≤ a1 ≤ n(r1w) ≤ (k1 − 1)(ε− 1). It follows that k1k2 − 2k2 + 1 ≤ k1ε, and so k2 − 2 + 1 k1 ≤ k2 − 2k2 k1 + 1 k1 ≤ ε < k2 − 1, contradicting the fact that ε is an integer. We now give an upper bound for a1. Lemma 4.3. Assume that g = 4 and ε ≥ 1. Let α = h(sn−1) and β = h(rm−1). Then a1 ≤ (α− 1)(k1 − 1) + ε+ (k2 − α)(ε− α− σ + 1). (4.1) and a1 ≤ (β − 1)(k2 − 1) + ε+ (k1 − β)(ε− β − γ + 1). (4.2) Proof. We prove inequality (4.1). The proof of inequality (4.2) is similar. Let {w1, . . . , wα} = Γ(sn−1) ∩D23 . We estimate n(sn−1w1). To do this we split the girth cycles (w1, sn−1, x, y, w1) into two types depending on the vertex x. We say that the girth cycle is of type 1 if x ∈ {w2, . . . , wα}, and of type 2 if x ∈ {u} ∪D21 . By Lemma 4.1(i) there are at most (α− 1)(k1− 1) girth cycles of type 1. To estimate the number of girth cycles of type 2, we further split these girth cycles into two subfamilies depending on the vertex y. Let us say that the girth cycle (w1, sn−1, x, y, w1) with x ∈ {u} ∪D21 is of type 2a if y ∈ D12 , and of type 2b if y ∈ D32 . If the girth cycle is of type 2b, then x ∈ D21 , and so by Lemma 4.1(ii) there are at most ε such girth cycles. To estimate the number of girth cycles of type 2a, observe that sn−1 has k2 − α neighbours in {u} ∪D21 , and that w1 has at most ε− α− σ + 1 neighbours in D12 \{sn−1}. This shows that the number of girth cycles of type 2a is at most (k2−α)(ε− α− σ + 1). As a1 ≤ n(sn−1w1), the result follows. Lemma 4.4. Assume that g = 4 and ε ≥ 1. Then ε = k2 − 2 and k2 − 1 ≥ 2k1/3. Proof. As in Lemma 4.3, let α = h(sn−1). Then, by Lemmas 3.8 and 4.3, we get that (k1 − 1)(k2 − 1)− σ(k1 − 1)− ε ≤ (α− 1)(k1 − 1) + ε+ (k2 − α)(ε− α− σ + 1). Rearranging the above inequality we find this is equivalent to (k1 − 1)(k2 − 1) ≤ (k1 − k2 − 1 + α)(α+ σ − 1) + ε(k2 − α+ 2). (4.3) Taking into account that α+ σ ≤ ε and that α ≥ 1, inequality (4.3) implies that (k1 − 1)(k2 − 1) ≤ (k1 + 1)ε− k1 + 1 + k2 − α ≤ (k1 + 1)ε− k1 + k2, Gy. Kiss et al.: On girth-biregular graphs 521 and so ε ≥ (k1 − 1)(k2 − 1) + k1 − k2 k1 + 1 = k2 − 3k2 − 1 k1 + 1 . (4.4) As k1 ≥ k2, the above inequality yields ε ≥ k2 − 3k1 − 1 k1 + 1 = k2 − 3 + 4 k1 + 1 > k2 − 3. Recall that ε < k2 − 1 by assumption, and so ε = k2 − 2 as claimed. Plugging ε = k2 − 2 into (4.4) we easily get that k2 − 1 ≥ 2k1/3. Theorem 4.5. Assume that g = 4. Then ε = 0 and Γ is the complete bipartite graph Kk1,k2 . Proof. Suppose on the contrary that ε ≥ 1. Recall that ε = k2 − 2. As in Lemma 4.3, let β = h(rm−1). Then, by Lemmas 3.8 and 4.3, we get that (k1 − 1)(k2 − 1)− γ(k2 − 1)− ε ≤ (β − 1)(k2 − 1) + ε+ (k1 − β)(ε− β − γ + 1). Rearranging the terms of the above inequality we get (k1 − 1)(k2 − 1) ≤ ε(k1 − β + 2) + (β + γ − 1)(k2 − k1 + β − 1). (4.5) If β = 1, then inequality (4.5) together with ε = k2 − 2 yields k1 − 1 ≤ γ(k2 − k1). But this is a contradiction as k1 > k2 > 0. If k2 − k1 + β − 1 ≤ 0, then inequality (4.5) together with ε = k2 − 2 and β ≥ 2 yields (k1 − 1)(k2 − 1) ≤ (k2 − 2)(k1 − β + 2) ≤ (k2 − 2)k1, implying k1 ≤ k2 − 1, a contradiction. Therefore, we have that k2 − k1 + β − 1 > 0 and β ≥ 2. Recall that β + γ ≤ ε = k2 − 2, and so inequality (4.5) gives us (k1−1)(k2−1) ≤ ε(k1−β+2)+(ε−1)(k2−k1+β−1) = (k2−2)(k2+1)−k2+k1−β+1. It follows that 2 ≤ β ≤ k22 − k2 − 2 + 2k1 − k1k2, or k1(k2 − 2) ≤ k22 − k2 − 4. As k1 ≥ k2 +1 this yields −2 ≤ −4, a contradiction. This shows that ε = 0 as claimed. It is now easy to see that Γ is isomorphic to the complete bipartite graph Kk1,k2 . 5 The case g = 2d ≥ 8, where d is even In this section we study girth-biregular graphs with girth g = 2d ≥ 8, d even. Throughout this section we will use Notation 3.5. Assume that g = 2d ≥ 8. For every z ∈ D21 we define β(z) = ∑ r∈Ddd−1∩Γd−2(z) h(r). 522 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 Note that for z ∈ D21 we have |Ddd−1 ∩ Γd−2(z)| = (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 and that for z, z′ ∈ D21 (z ̸= z′), the sets Ddd−1 ∩ Γd−2(z) and Ddd−1 ∩ Γd−2(z′) are disjoint as the girth of Γ is 2d. Therefore,∑ z∈D21 β(z) = ∑ r∈Ddd−1 h(r) = ε. (5.1) In particular, β(z) ≤ ε. Recall also that for an edge e of Γ we denoted by n(e) the number of girth cycles passing through e. Lemma 5.1. Assume that g = 2d ≥ 8 and ε ≥ 1. Then a1 ≥ (k1 − 1)d/2(k2 − 1)d/2 − k2ε. Proof. Abbreviate ℓ = (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2. Pick z ∈ D21 with n(vz) = a1 and let w1, w2, . . . , wℓ be the vertices of Ddd−1 ∩ Γd−2(z). For 1 ≤ j ≤ ℓ consider the 2d-cycles of the form (v, z, . . . , wj , b, r, r′, . . .) with b ∈ Dd−1d , where (v, z, . . . , wj) is the unique path from v to wj of length d− 1. Observe that for fixed wj and r, there is only one such cycle (recall that as g ≥ 8, wj and r have a unique common neighbour), and that for fixed wj and b, we could choose r in k2 − 1− h(b) different ways. Therefore, a1 = n(vz) ≥ ℓ∑ j=1 ∑ b∈Γ(wj)∩Dd−1d (k2 − 1− h(b)) = ℓ∑ j=1 ∑ b∈Γ(wj)∩Dd−1d (k2 − 1)− ℓ∑ j=1 ∑ b∈Γ(wj)∩Dd−1d h(b). (5.2) Furthermore, observe that for a fixed wj we could choose b in (k1 − 1 − h(wj)) different ways, and so ℓ∑ j=1 ∑ b∈Γ(wj)∩Dd−1d (k2−1) = (k2−1) ℓ∑ j=1 (k1−1−h(wj)) = ℓ(k1−1)(k2−1)−(k2−1)β(z). Finally, the sets Γ(wj)∩Dd−1d and Γ(wℓ)∩D d−1 d are disjoint if j ̸= ℓ (otherwise we would get a cycle of length 2d− 2), and so ℓ∑ j=1 ∑ b∈Γ(wj)∩Dd−1d h(b) ≤ ∑ b∈Dd−1d h(b) = ε. This, together with β(z) ≤ ε, shows that a1 = n(vz) ≥ ℓ(k1− 1)(k2− 1)− (k2− 1)β(z)− ε ≥ (k1− 1)d/2(k2− 1)d/2−k2ε. Lemma 5.2. Assume that g = 2d ≥ 8 and ε ≥ 1. Then a1 < (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 ( k1ε− k1 + 2 ) . Gy. Kiss et al.: On girth-biregular graphs 523 Proof. Let D = d−1⋃ i=0 ( Dii+1 ∪Di+1i ) . For vertices x, y ∈ D, let dD(x, y) denote the distance between x and y in the subgraph Γ[D], that is, in the subgraph of Γ, that is induced by D. Observe that dD(x, y) ≤ 2d − 1 for all x, y ∈ D. Pick a vertex r ∈ Ddd−1 with h(r) ≥ 1 and abbreviate α = h(r). Pick w ∈ Γ(r)∩D d+1 d and consider the set C of 2d-cycles (x0 = w, x1 = r, x2, . . . , x2d−1, w) through wr. Note that, as w ̸∈ D at most 2d − 2 edges of such a cycle have both endpoints in D. For 1 ≤ i ≤ 2d − 1 let Ci denote the subset of C defined as follows. A cycle (x0 = w, x1 = r, x2, . . . , x2d−1, w) is an element of Ci if and only if {x1, . . . , xi} ⊆ D and xi+1 ̸∈ A, where the addition in subscripts is computed modulo 2d. For example, cycles in C1 are those 2d-cycles (x0 = w, x1 = r, x2, . . . , x2d−1, w) , for which x2 ̸∈ D, while cycles in C2d−1 are those for which {x1, x2, . . . , x2d−1} ⊆ D. Note that the sets Ci are pairwise disjoint, and so a1 ≤ n(wr) ≤ |C1|+ |C2|+ · · ·+ |C2d−1|. Let us now estimate the above sum. To do this we introduce the following notation. For i ∈ {1, 3, . . . , 2d− 1} we define εi = ∑ b∈Dd−1d dD(r,b)=i h(b). Note that as Γ[D] is bipartite with diameter at most 2d− 1, we have that ε1 + ε3 + · · ·+ ε2d−1 = ε. We also define κ = |Γ(w) ∩ (Ddd−1 \ {r})| = |Γ(w) ∩Ddd−1| − 1. Note that α+ κ ≤ ε. Consider a 2d-cycle (x0 = w, x1 = r, x2, . . . , x2d−1, w) ∈ C1. Observe that there are α − 1 choices for x2. For each such choice of x2, there are, by Lemma 3.1, at most (k1 − 1)(d−2)/2(k2 − 1)d/2 girth cycles containing both edges wr and rx2. Therefore, |C1| ≤ (α− 1)(k1 − 1)(d−2)/2(k2 − 1)d/2. Consider a 2d-cycle (x0 = w, x1 = r, x2, . . . , x2d−1, w) ∈ C2 ∪ C4 ∪ · · · ∪ C2d−2. Assume that this cycle is an element of C2j (1 ≤ j ≤ d − 1). Observe that in this case we have that x2j ∈ Dd−1d and that dD(r, x2j) = 2j − 1 (otherwise there would be a cycle of length less than 2d). Therefore, we could choose an edge x2jx2j+1 in ε2j−1 different ways. For each such choice of an edge x2jx2j+1, there are, by Lemma 3.1, at most (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 girth cycles containing edges wr and x2jx2j+1, and so |C2|+ |C4|+ · · ·+ |C2d−2| ≤ (ε1 + ε3 + · · ·+ ε2d−3)(k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 = ε(k1 − 1)(d−2)/2(k2 − 1)(d−2)/2. 524 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 Consider a 2d-cycle (x0 = w, x1 = r, x2, . . . , x2d−1, w) ∈ C3 ∪ C5 ∪ · · · ∪ C2d−3. If this cycle is an element of C2j+1 (1 ≤ j ≤ d−2), then it is easy to see that x2j+1 ∈ Ddd−1, and so x2j+2 ∈ Dd+1d \ {w}. Therefore, there are at most ε − κ − α choices for an edge x2j+1x2j+2. For each such choice there are, by Lemma 3.1, at most (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2 girth cycles containing edges wr and x2j+1x2j+2, and so |C3|+ |C5|+ · · ·+ |C2d−3| ≤ (ε− κ− α)(k1 − 1)(d−4)/2(k2 − 1)(d−2)/2. Finally, consider a 2d-cycle (x0 = w, x1 = r, x2, . . . , x2d−1, w) ∈ C2d−1. Note that we have at most k1−α choices for a vertex x2. For each choice of vertices x2, x3, . . . , xi−1, where i ≤ d, we have at most k1 − 1 choices for vertex xi if i is even, and k2 − 1 choices for xi if i is odd. Therefore, there are at most (k1 − α)(k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 choices for vertices x2, x3, . . . , xd. On the other hand, there are at most κ choices for a vertex x2d−1. For each such choice of vertices x2, x3, . . . , xd and x2d−1, there is at most one girth cycle containing the edges wr, rx2, x2x3, . . . xd−1xd and x2d−1w. Therefore, |C2d−1| ≤ κ(k1 − α)(k1 − 1)(d−2)/2(k2 − 1)(d−2)/2. To further estimate the sum |C1|+ |C2|+ · · ·+ |C2d−1|, we first note that |C1|+ |C2d−1| ≤ (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2( (α− 1)(k1 − 1)(k2 − 1) + κ(k1 − α)(k1 − 1) ) < (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2( (α− 1)(k1 − 1)2 + κ(k1 − α)(k1 − 1) ) = (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2( (α− 1 + κ)(k1 − 1)2 − κ(α− 1)(k1 − 1) ) ≤ (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2(α− 1 + κ)(k1 − 1)2 ≤ (k1 − 1)d/2(k2 − 1)(d−2)/2(ε− 1), while |C2|+ |C3|+ · · ·+ |C2d−2| ≤ (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2(ε(k1 − 1) + (ε− κ− α)) ≤ (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2 ( ε(k1 − 1) + ε− 1 ) = (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2(k1ε− 1). Therefore, a1 ≤ n(wr) ≤ |C1|+ |C2|+ · · ·+ |C2d−1| ≤ (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2 ( (ε− 1)(k1 − 1)2 + k1ε− 1 ) = (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2 ( ε(k21 − k1) + ε− (k1 − 1)2 − 1 ) < (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2 ( ε(k21 − k1) + (k2 − 1)− (k1 − 1)2 ) < (k1 − 1)(d−4)/2(k2 − 1)(d−2)/2 ( ε(k21 − k1) + (k1 − 1)− (k1 − 1)2 ) = (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 ( k1ε− k1 + 2 ) . The result follows. Gy. Kiss et al.: On girth-biregular graphs 525 Theorem 5.3. Assume that g = 2d ≥ 8 and d is even. Then ε = 0 and Γ is the incidence graph of a finite thick generalized d-gon, hence either d = 4 or d = 8. Proof. Suppose first that ε is positive. By Lemma 5.1 and 5.2 we have (k1 − 1)d/2(k2 − 1)d/2 − k2ε ≤ a1 < (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 ( k1ε− k1 + 2 ) . This implies k2 − 1 > ε > (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2(k1k2 − k2 − 1) k2 + k1(k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 > (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2(k1k2 − k2 − 1) k1 ( 1 + (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 ) = k2 − 2 + (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2(2k1 − k2 − 1)− k1(k2 − 2) k1 ( 1 + (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2 ) As k1(k2 − 2) < (k1 − 1)(k2 − 1) < (k1 − 1)(d−2)/2(k2 − 1)(d−2)/2(2k1 − k2 − 1), the above inequality implies k2 − 1 > ε > k2 − 2, contradicting the fact that ε is an integer. Therefore, ε = 0. 6 The case g = 2d, where d is odd In this section we consider the case g = 2d with d odd, in particular the case g = 6 when we provide a characterization of affine planes. Unfortunately, the method we applied in the proof of Lemma 5.2 for giving an upper estimate on b1 does not work for odd d, but we can calculate the exact value of b1 if ε = 1. Throughout this section we will use Notation 3.5. Theorem 6.1. Assume that d is odd and suppose that ak1 = bk2 = M − 1. Then b1 = M − k2 + 1 and b2 = · · · = bk2 = M − 1. Proof. Pick adjacent vertices u ∈ A, v ∈ B such that n(uv) = ak1 = bk2 = M − 1. Let Dij denote D i j(u, v) and D = d−1⋃ i=0 ( Dii+1 ∪Di+1i ) . For vertices x, y ∈ D, let dD(x, y) denote the distance between x and y in the subgraph Γ[D], that is, in the subgraph of Γ, that is induced by D. Observe that dD(x, y) ≤ 2d − 1 for all x, y ∈ D. By Proposition 2.6(vi) and (vi) we have that |Dd−1d | = |D d d−1| = (k1 − 1)(g−2)/4(k2 − 1)(g−2)/4 = M k2 − 1 , and there are M − 1 edges between Dd−1d and Ddd−1. Hence all but one vertices in Ddd−1 have k2 − 1 neighbours in Dd−1d . Let p ∈ Ddd−1 denote the unique vertex which has only k2 − 2 neighbours in Dd−1d . 526 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 We claim that all but one vertices in Dd−1d have k2 − 1 neighbours in Ddd−1, too. Let x be any vertex in Dd−1d . Then for each vertex y ∈ D21 there is at most one vertex z ∈ Ddd−1 ∩ Γ(x) so that d(y, z) = d − 2, because otherwise a cycle of length 2(d − 1) would appear. Thus |Γ(x) ∩Ddd−1| ≤ |D21| = k2 − 1. This implies, by the pigeonhole principle, that there is a unique vertex r ∈ Dd−1d which has only k2− 2 neighbours in Ddd−1. Then r has one neighbour in D d−2 d−1 and it has k1− k2+1 neighbours outside D. Now, let w be an arbitrary vertex in D21 and let S = D d d−1 \D d−2 d−1(w, v). Then Ddd−1(w, v) = D d−2 d−1 ∪ S. We now describe the set Dd−1d (w, v). Observe that Dd−1d (w, v) ⊆ D d−1 d ∪ {p1}, (6.1) where p1 is the unique neighbour of p outside D. There are two possibilities we have to consider, namely either w is the unique vertex of D21 for which dD(p, w) = d − 2, or dD(p, w) = d. Let us first consider the case dD(p, w) = d − 2. Note that in this case p1 ∈ Dd−1d (w, v), so there is a unique vertex w1 ∈ D d−1 d which is not contained in Dd−1d (w, v). Observe that every vertex from D d−1 d , which has k2 − 1 neighbours in Ddd−1, is at distance d− 1 from w, and so w1 = r. Therefore (6.1) implies Dd−1d (w, v) = ( Dd−1d \ {r} ) ∪ {p1}. We now count the number of neighbours between Dd−1d (w, v) and D d d−1(w, v). Recall that each vertex from Dd−1d has a unique neighbour in D d−2 d−1 and that each vertex from Dd−1d \ {r} has k2 − 1 neighbours in Ddd−1. Pick x ∈ D d−1 d \ {r}. As x ∈ D d−1 d (w, v), x has at least one neighbour in Ddd−1\S. On the other hand, if x has more than one neighbour in Ddd−1 \ S, then this would imply a cycle of length 2(d − 1), a contradiction. Using the above observations we now have n(vw) = ( |Dd−1d | − 1 ) (k2 − 1) = ( (k1 − 1)(d−1)/2(k2 − 1)(d−1)/2 − 1 ) (k2 − 1) = M − k2 + 1. In the case when dD(p, w) = d − 2 we have that d(w, p1) = d + 1 (note that p is the only neighbour of p1 in D), and so by (6.1) we have Dd−1d (w, v) = D d−1 d . Observe also that |S| = |Ddd−1| − |D d−2 d−1(w, v)| = (k1 − 1)(d−1)/2(k2 − 1)(d−3)/2(k2 − 2). Similar arguments as in the previous case now show that n(vw) = |Dd−1d |+ |S|(k2 − 1)− 1 = (k1 − 1)(d−1)/2(k2 − 1)(d−1)/2 + (k1 − 1)(d−1)/2(k2 − 1)(d−1)/2(k2 − 2)− 1 = M − 1. This proves the statement. Gy. Kiss et al.: On girth-biregular graphs 527 Theorem 6.2. Assume that d is odd and k2 does not divide k1. If ak+1 = bk = M − ε for a non-negative integer ε ≤ 1, then ε = 0 and a1 = · · · = ak+1 = b1 = · · · = bk = M. Proof. We first assume ε = 1 and derive a contradiction. If ε = 1, then it follows from Theorem 6.1 that the signature of any vertex from B is (M − k2 + 1,M − 1, . . . ,M − 1). Now Proposition 3.2 yields that the signature of any vertex from A is (M−k2+1, . . . ,M− k2+1,M−1, . . . ,M−1). Let a = M−k2+1 and let aA and aB be as in Proposition 3.3. Observe that aB = 1 and so we have k2aA = k1 by Proposition 3.3. Hence k1 is divisible by k2, a contradiction. Therefore ε = 0 and the result now follows from Theorem 3.4. In particular, we consider the case k1−1 = k2 = k and d = 3. Then k1k2−k1+1 = k2 and it is well-known that a 2−(k2, k, 1) design is a finite affine plane of order k. Combining Theorems 3.4(vii) and 6.2 we get the following characterization. Corollary 6.3. Assume that k1 − 1 = k2 = k and that d = 3. If ak+1 = bk = M − ε for a non-negative integer ε ≤ 1, then ε = 0 and Γ is the incidence graph of a finite affine plane of order k. 7 Examples In this section we provide some examples where ak1 is close to the upper bound given in Theorem 3.4. In all cases, the signatures of the points are constants, hence each edge is contained in the same number of girth cycles. So our examples are edge-girth-regular graphs, too. Let us start with the g = 4 case. Example 7.1. Let f1 > f2 ≥ 1 and h > 2 be integers and consider the complete bipartite graph Γ′ = Kf1h,f2h with bipartition A and B. Label the vertices so that A = f1⋃ i=1 {u1,i, u2,i, . . . , uh,i} , B = f2⋃ j=1 {v1,j , v2,j , . . . , vh,j} . Let Γ denote a graph that is obtained from Γ′ by deleting all edges of the form uℓ,ivℓ,j , where ℓ ∈ {1, 2, . . . , h}, i ∈ {1, 2, . . . , f1} and j ∈ {1, 2, . . . , f2}. Then Γ is a bipartite biregular graph with g = 4, k1 = f2(h− 1) and k2 = f1(h− 1). Take any edge e = uℓ1,ivℓ2,j in Γ. Then ℓ1 ̸= ℓ2, and there are ((f2(h − 1) − 1) (f1(h − 1) − 1) 3-arcs of Γ which contain e. Let us now count how many of these 3-arcs are not contained in a 4-cycle. Let A = vℓ′,j′uℓ1,ivℓ2,juℓ′′,i′′ be any 3-arc containing edge e. Note that ℓ′ ̸= ℓ1 and ℓ′′ ̸= ℓ2. Then A is not contained in a 4-cycle if and only if vertices vℓ′,j′ and uℓ′′,i′′ are not adjacent in Γ, which happens if and only if ℓ′ = ℓ′′. As ℓ′ ̸= ℓ1 and ℓ′′ ̸= ℓ2, there are h−2 choices for ℓ′ = ℓ′′, hence there are f1f2(h−2) 3-arcs containing e, that are not contained in a 4-cycle. So the number of girth cycles through e in Γ is ((f2(h− 1)− 1) (f1(h− 1)− 1)− f1f2(h− 2). It follows that Γ is girth-biregular with a1 = · · · = ak1 = b1 = · · · = bk2 = (k1− 1)(k2− 1)− f1f2(h− 2) = M − f1f2(h− 2). For g = 6 we follow the examples of the paper [1]. 528 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 Example 7.2. Take an affine plane of order q and remove i parallel classes. Consider the incidence graph of this structure. The lines still have size q and the points have degree q+1−i, so it is a bipartite biregular graph with valencies q and q+1−i. To count the girth cycles containing the edge corresponding to an incident point-line pair (e0, P0), we have to choose a point P0 ̸= P1 ∈ e0, and a line e0 ̸= e1 through P0 and complete it to a girth cycle (of length 6) by choosing a point P0 ̸= P2 ∈ e1 and a line e2 joining P1 and P2. There are q − 1 ways to choose P1 and q − i ways of choosing e1. For e2 we have to choose a line different from e1, not parallel to e0, so we have (q− 1− i) possibilities, since the point P2 will just be the unique point of e0∩e2. So, in total there are M ′ = (q−1)(q− i)(q−1− i) girth cycles through the edge (e0, P0). In particular, when we have an affine plane of order q, its incidence graph is a bipartite biregular graph with valencies q+1 and q, and we have M = (q−1)2q girth cycles through an edge. If there is an affine plane of order q + 1 as well, then removing i = 2 parallel classes will also give us a bipartite biregular graph with valencies q+1 and q and this graph will have M ′ = q(q − 1)(q − 2) = M − q(q − 1) girth cycles through every edge. Another construction from the paper [1] is the following. Example 7.3. Let us consider a Steiner system on v points and line size k. Delete a point P ∗ and all the lines through the deleted point. The incidence graph of the resulting structure will be a bipartite biregular graph with valencies k and r−1, again with r = (v−1)/(k−1). One can more or less copy the argument in the previous example: using the same notation, the point P1 can be chosen in (k−1) ways. Now consider the line e∗ in the original Steiner system that joins P1 and P ∗. If the line e1 intersects e∗, then we have (k − 2) choices for P2 and e2, and there are (k−2) such lines in the original Steiner system. So, this case gives (k− 1)(k− 2)2 girth cycles. There remain (r− 2)− (k− 2) = r− k lines through P0, not intersecting e∗. If e1 is one of them, then there are (k−1) ways to extend it to a girth cycle. This is (k− 1)2(r− k) possibility, so in total we have (k− 1)((k− 2)2 + (r− k)(k− 1)) girth cycles containing the edge (e0, P0). It is easy to extend Example 7.2 to resolvable Steiner systems. Example 7.4. Consider a resolvable Steiner system and denote by v the number of points, by r the degrees of points, where r = (v − 1)/(k − 1). In this case k divides v, and the original design will have (k − 1)2(r − 1) girth cycles through any edge. If we remove i parallel classes of lines, then the incidence graph of the resulting structure will have degrees k and r − i). For determining the number of girth cycles containing an edge start from an incident point-line pair (P0, e0) as before. Take a point P1 on e0 and let U be the set of points which are on the lines through P1 that belong to the deleted parallel classes. This implies that |U | = i(k − 1). Let rj , j = 0, . . . , k − 1, be the number of lines through P0 which intersect U in exactly j points. Clearly, we have ∑ j rj = r − 1, and∑ j jrj = |U | = i(k − 1). On a line ℓ through having j points in U , we can choose the point P2 of the girth cycle in (k − 1− j) ways. This way we get in total k−1∑ j=0 (k − 1− j)rj = (k − 1)(r − 1)− i(k − 1) girth cycles for a given choice of P1, so the total number of girth cycles will be (k − 1)2((r − 1)− i). For small i this is close to our upper bound. Gy. Kiss et al.: On girth-biregular graphs 529 In particular, we mention two examples arising from higher dimensional finite spaces. 1. Let n = 2m + 1. Remove the qm + 1 elements of a line spread from PG(n, q) and denote the correponding point-line incidence graph by Γ. Then Γ is a girth-biregular bipartite graph with g = 6, k1 = q2m + · · ·+ q and k2 = q + 1 and its signature is a1 = · · · = ak1 = b1 = · · · = bk2 = q2(q2m + · · ·+ q − 2) = M − q2. 2. Let us remove the qn−1 elements of a class of parallel lines from AG(n, q) and denote the corresponding point-line incident graph by Γ. Then Γ is a girth-biregular bipartite graph with g = 6, k1 = qn−1 + · · ·+ q and k2 = q and its signature is a1 = · · · = ak1 = b1 = · · · = bk2 = (q− 1)2(qn−1 + · · ·+ q− 2) = M − (q− 1)2. In both cases the magnitude of ε is only k2/(n−1)1 . In the case g = 8 our examples come from incidence graphs of generalized quadrangles. For a detailed descriptions of generalized quadrangles, their ovoids and spreads, we refer the reader to the book of Payne and Thas [13]. Example 7.5. Let G′ = (P,L, I) be a generalized quadrangle of order (s, t) and Γ′ be the Levi graph of G′. Suppose that G′ admits a spread S (a set of st + 1 lines, no two of which intersect). Delete the lines of S. Then the Levi graph Γ of G = (P,L \ S, I) is a bipartite graph with bipartition |A| = (s + 1)(st + 1) and |B| = t(st + 1), valencies s + 1 and t and g = 8. We claim that it is also girth-biregular with a1 = · · · = as+1 = b1 = · · · = bt = s2(t2 − 3t+ 2) = M − s2(t− 1). Dually, if G′ admits an ovoid O (a set of st+ 1 points, no two of which are collinear), then the Levi graph Γ of G = (P \ O,L, I) is a girth-biregular graph with valencies s and t+ 1, and a1 = · · · = as+1 = b1 = · · · = bt = t2(s2 − 3s+ 2) = M − t2(s− 1). In G for any incident point-line pair (P, ℓ) there are (t − 1)s points in P which are collinear with P but are not incident with ℓ, and there are s(t−1) lines in which meet ℓ but are not incident with P . Let R be one of these points and e be one of these lines. Then there is a unique point-line pair (T, f) in G′ so that R I f IT I e. Thus in Γ′ there are s2(t− 1)2 girth cycles through the edge which corresponds to the pair (P, ℓ). For a fixed R there is a unique element f ∈ S through R. All the s other points on f determines a unique 8-cycle which contains (P, ℓ). No two elements of S intersect, hence there are (t− 1)s · s deleted 8-cycles. Thus in Γ′ the total number of girth cycles through the edge corresponding to (P, ℓ) is s2(t− 1)2 − s(t− 1)s = s2(t2 − 3t+ 2) = s2(t− 1)(t− 2). Among the known generalized quadrangles only a few admit a spread or an ovoid. In particular, the classical generalized quadrangle Q(5, q) admits a spread. In this case Γ has valencies q+1 and q2, and the number of girth cycles through every edge is q2(q2−1)(q2− 2) = M − q2(q2 − 1). So the magnitude of ε is M2/3. 530 Ars Math. Contemp. 23 (2023) #P4.01 / 509–530 ORCID iDs György Kiss https://orcid.org/0000-0003-3312-9575 Štefko Miklavič https://orcid.org/0000-0002-2878-0745 Tamás Szőnyi https://orcid.org/0000-0001-7184-8496 References [1] G. Araujo-Pardo, R. Jajcay, A. Ramos-Rivera and T. Szőnyi, On a relation between bipartite biregular cages, block designs and generalized polygons, J. Comb. Des. 30 (2022), 479–496, doi:10.1002/jcd.21836, https://doi.org/10.1002/jcd.21836. [2] P. J. Cameron and J. H. van Lint, Designs, graphs, codes and their links, volume 22 of London Mathematical Society Student Texts, Cambridge University Press, Cambridge, 1991, doi:10. 1017/cbo9780511623714, https://doi.org/10.1017/cbo9780511623714. [3] C. J. Colbourn and J. H. Dinitz (eds.), The CRC Handbook of Combinatorial Designs, CRC Press Series on Discrete Mathematics and its Applications, CRC Press, Boca Raton, FL, 1996, doi:10.1201/9781420049954, https://doi.org/10.1201/9781420049954. [4] G. Exoo and R. Jajcay, Dynamic cage survey, Electron. J. Comb. DS16 (2008), 48, doi:10. 37236/37, https://doi.org/10.37236/37. [5] W. Feit and G. Higman, The nonexistence of certain generalized polygons, J. Algebra 1 (1964), 114–131, doi:10.1016/0021-8693(64)90028-6, https://doi.org/10.1016/ 0021-8693(64)90028-6. [6] S. Filipovski, A. R. Rivera and R. Jajcay, On biregular bipartite graphs of small excess, Discrete Math. 342 (2019), 2066–2076, doi:10.1016/j.disc.2019.04.004, https://doi.org/10. 1016/j.disc.2019.04.004. [7] Z. Füredi and M. Simonovits, The history of degenerate (bipartite) extremal graph problems, in: Erdös Centennial, János Bolyai Math. Soc., Budapest, volume 25 of Bolyai Soc. Math. Stud., pp. 169–264, 2013, doi:10.1007/978-3-642-39286-3\ 7, https://doi.org/10.1007/ 978-3-642-39286-3_7. [8] R. Jajcay, Gy. Kiss and Š. Miklavič, Edge-girth-regular graphs, European J. Comb. 72 (2018), 70–82, doi:10.1016/j.ejc.2018.04.006, https://doi.org/10.1016/j.ejc. 2018.04.006. [9] Gy. Kiss, Š. Miklavič and T. Szőnyi, A stability result for girth-regular graphs with even girth, J. Graph Theory 100 (2022), 163–181, doi:10.1002/jgt.22770, https://doi.org/10. 1002/jgt.22770. [10] Gy. Kiss and T. Szőnyi, Finite Geometries, CRC Press, Boca Raton, FL, 2020. [11] H. V. Maldeghem, Generalized Polygons, Monographs in Mathematics, Birkhäuser Verlag, Basel, 1998, doi:10.1007/978-3-0348-8827-1, https://doi.org/10.1007/ 978-3-0348-8827-1. [12] M. Miller and J. Širáň, Moore graphs and beyond: a survey of the degree/diameter problem, Electron. J. Comb. DS14 (2005), 61, doi:10.37236/35, https://doi.org/10.37236/ 35. [13] S. E. Payne and J. A. Thas, Finite Generalized Quadrangles, EMS Series of Lectures in Math- ematics, European Mathematical Society (EMS), Zürich, 2nd edition, 2009, doi:10.4171/066, https://doi.org/10.4171/066. [14] P. Potočnik and J. Vidali, Girth-regular graphs, Ars Math. Contemp. 17 (2019), 349– 368, doi:10.26493/1855-3974.1684.b0d, https://doi.org/10.26493/1855-3974. 1684.b0d. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.02 / 531–551 https://doi.org/10.26493/1855-3974.2698.6fe (Also available at http://amc-journal.eu) An extension of the Erdős-Ko-Rado theorem to uniform set partitions Karen Meagher * , Mahsa N. Shirazi Department of Mathematics and Statistics, University of Regina, Regina, SK, S4S 0A2, Canada Brett Stevens † School of Mathematics and Statistics, Carleton University, Ottawa, ON, K1S 5B6, Canada Received 14 September 2019, accepted 25 December 22, published online 27 February 2023 Abstract A (k, ℓ)-partition is a set partition which has ℓ blocks each of size k. Two (k, ℓ)- partitions P and Q are said to be partially t-intersecting if there exist blocks Pi in P and Qj in Q such that |Pi ∩Qj | ≥ t. In this paper we prove a version of the Erdős-Ko- Rado theorem for partially 2-intersecting (k, ℓ)-partitions. In particular, we show for ℓ sufficiently large, the set of all (k, ℓ)-partitions in which a block contains a fixed pair is the largest set of 2-partially intersecting (k, ℓ)-partitions. For k = 3, we show this result holds for all ℓ. Keywords: Erdős-Ko-Rado Theorem, uniform set partitions, ratio bound, clique, coclique, quotient graphs. Math. Subj. Class. (2020): 05E30, 05C50, 05C25 1 Introduction In 1961, Erdős, Ko, and Rado proved that if F is a t-intersecting family of k-subsets of {1, 2, . . . , n}, then ( n−t k−t ) is a tight upper bound on the size of F , provided that n is suffi- ciently large [7]. This result has motivated consideration of “intersecting” families of many *Corresponding author. Research supported by NSERC Discovery Research Grant, Application No.: RGPIN- 2018-03952. †Research supported by NSERC Discovery Research Grant, Application No.: RGPIN-2017-06392. E-mail addresses: karen.meagher@uregina.ca (Karen Meagher), mahsa.nasrollahi@gmail.com (Mahsa N. Shirazi), brett@math.carleton.ca (Brett Stevens) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 532 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 other combinatorial objects using diverse proof techniques and has developed into an ac- tive and broad area of research. There are many recent results giving analogs of the EKR theorem; see, for example, [9, 13, 16, 20, 26] or [12] and the references within. In this work, we prove an extension of the EKR theorem to systems of uniform set partitions. A (k, ℓ)-partition is a set partition of {1, 2, . . . , kℓ} with exactly ℓ blocks each of size k. These are also called uniform set partitions. We use Uk,ℓ to denote the set of all (k, ℓ)- partitions, and uk,ℓ = |Uk,ℓ|. It is easy to see that uk,ℓ = 1 ℓ! ( kℓ k )( kℓ− k k )( kℓ− 2k k ) · · · ( k k ) = (kℓ)! (k!)ℓℓ! . (1.1) In [8], Erdős and Székely considered different types of intersection for partitions. In one of these types, and the one we consider here, two partitions P and Q are intersecting in a pair if there exist blocks Pi in P , and Qj in Q such that |Pi ∩Qj | ≥ 2. In [20], Meagher and Moura generalized this definition: two partitions P and Q are partially t-intersecting if there exist Pi in P , and Qj in Q such that |Pi ∩Qj | ≥ t. The work of Meagher and Moura is different than that of Erdős and Székely since only uniform partitions are considered in [20]. A set of partitions is partially t-intersecting if the partitions in the set are pairwise par- tially t-intersecting. Meagher and Moura [20] conjectured if P ⊂ Uk,ℓ is a set of partially t-intersecting partitions, with t ≤ k, then |P| ≤ ( kℓ−t k−t ) uk,ℓ−1. A set of this size can be formed by fixing a t-subset T and taking all (k, ℓ)-partitions with a block that contains T ; such a set is called canonically t-intersecting. Meagher and Moura further conjec- tured that only the canonically t-intersecting (k, ℓ)-partitions attain this maximum size. As pointed out by Brunk in [2], this conjecture additionally requires that k ≤ ℓ(t− 1), since if k > ℓ(t− 1), then any two (k, ℓ)-partitions are t-partially intersecting. If k = t = 2, then the (2, ℓ)-partitions are perfect matchings in the complete graph on 2ℓ vertices, and partially 2-intersecting is equivalent to intersecting (as sets of edges). The Meagher-Moura conjecture has been proven in this case in [13], so in this paper we only consider k ≥ 3. In particular, we prove the Meagher-Moura conjecture for t = 2 with k = 3 and all values of ℓ, and for all k ≥ 4, provided that ℓ is sufficiently large. Our approach is to define a graph in which the cocliques (also known as independent sets) are equivalent to partially 2-intersecting (k, ℓ)-partitions from Uk,ℓ. Then we use a version of the algebraic method from [13] to find the size of a maximum coclique in the graph. This is an approach that has been very effective in proving many EKR-type results, indeed it is the main topic of the book [12]. This method is particularly effective when considering intersecting permutations in groups and it has been applied to many families of groups, see for example [5, 6, 17, 21, 22, 23, 25]. 2 Overview of method In a graph X a clique is a set of vertices which induce a complete subgraph; and a coclique is a set of vertices which induce an empty subgraph. The size of a largest clique and a largest coclique are denoted by ω(X) and α(X), respectively. The adjacency matrix A(X) of X is a matrix in which rows and columns are indexed by the vertices in X and the (i, j)-entry is 1 if i and j are adjacent, and 0 otherwise. The eigenvalues of X refer to the eigenvalues of its adjacency matrix. We use 1 to denote the all-ones vector; for any d-regular graph, the all-ones vector is an eigenvector with eigenvalue d. K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 533 In general, finding the largest coclique of a graph X is known to be NP-hard, but the Delsarte-Hoffman (ratio) bound gives an upper bound on α(X). This bound is based on the ratio between the largest and the smallest eigenvalue of the adjacency matrix of the graph. A proof of this result can be found in [4] or in [12, Section 2.4], we also recommend Haemers’ paper [15] on the history of this bound. Theorem 2.1 (Delsarte-Hoffman bound [4]). Let A be the adjacency matrix for a d-regular graph X on vertex set V (X). If the least eigenvalue of A is τ , then α(X) ≤ |V (X)| 1− dτ . If equality holds for some coclique S with characteristic vector νS , then νS − |S| |V (X)| 1 is an eigenvector with eigenvalue τ . Define Xk,ℓ to be the graph with Uk,ℓ as its vertex set, in which two partitions P and Q are adjacent if every pair of blocks, one from P and one from Q, have at most 1 element in common. The group Sym(kℓ) acts transitively on the vertices of Xk,ℓ and preserves the edges. This means the Xk,ℓ is vertex transitive and regular. We will denote the degree by dk,ℓ, or simply d when the context is clear. A resolvable packing design on kℓ points with block size k and index λ = 1 is equiva- lent to a clique in this graph. Further, a resolvable balanced incomplete block design on kℓ points with block size k and index λ = 1, if it exists, gives a maximum clique. For any distinct i, j ∈ {1, . . . , kℓ}, let Si,j be the subset of partitions in Uk,ℓ for which the elements i and j are in the same block. Then Si,j is a coclique in the graph Xk,ℓ and the size of Si,j is 1 (ℓ− 1)! ( kℓ− 2 k − 2 )( kℓ− k k ) · · · ( k k ) . The main goal in this paper is to prove, using the ratio bound, that Si,j is a maximum coclique in Xk,ℓ. For the ratio bound to hold with equality, we need to prove if τ is the least eigenvalue of Xk,ℓ, then 1− dk,ℓ τ = uk,ℓ |Si,j | = kℓ− 1 k − 1 . Thus we need to prove two facts: first that τ = −dk,ℓ(k−1)k(ℓ−1) is an eigenvalue of Xk,ℓ; and second that τ is the least eigenvalue of Xk,ℓ. In Section 3, we show how the eigenvalues of Xk,ℓ are related to the irreducible charac- ters of Sym(kℓ), and we prove some bounds on the degrees of these irreducible characters. Next, in Section 4, we calculate three of the eigenvalues of Xk,ℓ; one of these eigenval- ues is the τ above. Next we prove if there is an eigenvalue of Xk,ℓ that is strictly smaller than τ , there is a function that is an upper bound on the eigenvalue’s multiplicity. In Sec- tion 6 we prove that this function is bounded by ( kℓ 3 ) − ( kℓ 2 ) for ℓ sufficiently large. This uses the result from Section 5, that the limit of ratio uk,ℓ/dk,ℓ is finite as ℓ goes to ∞. The bounds from Section 3 then prove that no such eigenvalues exist and this proves the 534 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 Meagher-Moura Conjecture with t = 2, for all values of k, provided that ℓ is sufficiently large. Finally, in Section 7, we prove a weaker bound on uk,ℓ/dk,ℓ when k = 3 that holds for all ℓ. Thus we prove the Meagher-Moura Conjecture for t = 2, k = 3 for all values of ℓ. 3 Representations of the symmetric group In this section we will explain the connection between the eigenvalues of the graph Xk,ℓ and the irreducible characters of the symmetric group. We also recall some results on the degree of the irreducible characters that are involved in the eigenvalues. For any character χ of Sym(n), we can consider its restriction to H ≤ Sym(n) which is denoted by res (χ)H . Similarly if χ is a character of H ≤ Sym(n), then its induced character on Sym(n) is denoted by ind (χ)Sym(n). The trivial character on a group H is denoted by 1H . The stabilizer of a partition in Uk,ℓ is the group Sym(k) ≀ Sym(ℓ) (this is called the wreath product of Sym(k) and Sym(ℓ)). The cosets Sym(kℓ)/(Sym(k) ≀ Sym(ℓ)) are in one-to-one correspondence with the partitions of Uk,ℓ. The action of Sym(kℓ) on the partitions is equivalent to the action of Sym(kℓ) on the cosets Sym(kℓ)/(Sym(k)≀Sym(ℓ)) and this action is clearly transitive. The permutation representation of this action is ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) . The module for this representation can be thought of as the vector space of length-uk,ℓ vectors with the characteristic vectors of P ∈ Uk,ℓ, denoted by vP , as its basis. The group Sym(kℓ) acts on this vector space by the action on the partitions, for any σ ∈ Sym(kℓ) the action is σ(vP ) = vPσ . This representation can be decomposed as the sum of irreducible representations of Sym(kℓ). If the multiplicity of each irreducible representation in the decomposition is equal to 1, then the representation is called multiplicity-free. In general, the group Sym(k) ≀ Sym(ℓ) is not multiplicity free in Sym(kℓ). In fact it is not multiplicity free unless k = 2, ℓ = 2, or (k, ℓ) is one of (3, 3), (4, 3), (5, 3) or (3, 4) [11]. 3.1 Orbital association scheme The set of orbitals of the action of a group G on a set Ω is the set of orbits of the action of G on Ω×Ω. Each orbital of Sym(kℓ) on Sym(kℓ)/(Sym(k) ≀Sym(ℓ)) can be represented by an object called a meet table. The meet table for two (k, ℓ)-partitions is a ℓ× ℓ array in which the (i, j)-entry is |Pi ∩Qj |. Two meet tables are isomorphic if one can be obtained from the other by permuting the rows and the columns. In [12, Section 15.4] it is shown that the set of non-isomorphic meet tables corresponds to the set of orbitals. For each orbital O there is a corresponding meet table M ; this means for P,Q ∈ Uk,ℓ the meet table of P and Q is M if and only if (P,Q) ∈ O. Further, each orbital can be represented as a uk,ℓ × uk,ℓ matrix, with the (P,Q)-entry equal to 1 if and only if the meet table of P and Q is isomorphic to the table representing the orbital. The set of these uk,ℓ × uk,ℓ-matrices of the orbitals forms an association scheme if and only if ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) is multiplicity-free. In general, these matrices form a homogeneous coherent configuration. K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 535 The graph Xk,ℓ is the union of the orbitals from the action of Sym(kℓ) on the cosets Sym(kℓ)/(Sym(k) ≀ Sym(ℓ)) that are represented by a meet table that has no entry greater than 1. This means for every permutation σ ∈ Sym(kℓ) its action on the partitions is an automorphism of Xk,ℓ. In particular, if Mσ is the permutation representation of σ, then Mσ−1 A(Xk,ℓ)Mσ = A(Xk,ℓ). Further, if v is any θ-eigenvector of Xk,ℓ, then Mσv is also a θ-eigenvector. This implies the eigenspaces of Xk,ℓ are invariant under the action of Sym(kℓ) and thus a union of irreducible modules in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) . We say that an eigenvalue θ belongs to a module if the module is a subspace of the θ- eigenspace. 3.2 Degree of the irreducible characters of Sym(kℓ) In this section we will give some results on the irreducible representations of Sym(n). We refer the reader to [24], or any similar reference on this topic, for details and background. It is well-known that the irreducible representations of Sym(n) correspond to integer parti- tions on n. We will use λ ⊢ n to indicate that λ is an integer partition of n, this means that λ = [λ1, λ2, . . . , λj ], each λi is an integer with ∑j i=1 λi = n. We will use χλ to represent the irreducible character of Sym(n) corresponding to the partition λ. From [13] we have a list of irreducible representations of the symmetric group with small degree. Lemma 3.1. For n ≥ 9, let χ be a character of Sym(n) with degree less than (n2−n)/2. If χλ is a constituent of χ, then λ is one of the following partitions of n: [n], [1n], [n− 1, 1], [2, 1n−2], [n− 2, 2], [2, 2, 1n−4], [n− 2, 1, 1], [3, 1n−3]. This proof uses the branching rule, which we state here. For a proof of this rule see [3, Corollary 3.3.11]. Lemma 3.2. Let λ ⊢ n, then res (χλ)Sym(n−1) = ∑ χλ− , where the sum is taken over all partitions λ− of n − 1 that have a Young diagram which can be obtained by the deletion of a single box from the Young diagram of λ. Further, ind (χλ) Sym(n+1) = ∑ χλ+ , where the sum is taken over partitions λ+ of n + 1 that have a Young diagram which can be obtained by the addition of a single box to Young diagram of λ. Using the same approach as the proof for Lemma 3.1 we can get a second family of irreducible characters with slightly larger, but still small degree. 536 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 Lemma 3.3. For n ≥ 13, let χ be an irreducible character of Sym(n) with degree less than ( n 3 ) − ( n 2 ) . If χλ is a constituent of χ, then λ is one of the following partitions of n: [n], [1n], [n− 1, 1], [2, 1n−2], [n− 2, 2], [2, 2, 1n−4], [n− 2, 1, 1], [3, 1n−3], [n− 3, 3], [2, 2, 2, 1n−6]. Proof. The hook length formula confirms that each of the 10 characters above have degree less than or equal to ( n 3 ) − ( n 2 ) . We prove this result by induction. For n = 13 and 14 this can be calculated directly using the GAP character table library [10]. We assume for n ≥ 14 that the lemma holds for n and n− 1, and we will prove that the lemma holds for n+ 1. Assume that χ is an irreducible character of Sym(n+ 1) that has dimension less than( n+ 1 3 ) − ( n+ 1 2 ) = (n+ 1)n(n− 4) 6 , but is not one of the ten irreducible characters listed in the statement of the lemma. We will show that such a χ cannot exist. If one of the ten irreducible characters of Sym(n) with degree less than ( n 3 ) − ( n 2 ) is a constituent of res (χ)Sym(n), then we can determine the possible constituents of χ with the branching rule. Constituent of res (χ)Sym(n) Constituents of χ [n] [n+ 1], [n, 1] [n− 1, 1] [n, 1], [n− 1, 2], [n− 1, 1, 1] [n− 2, 2] [n− 1, 2], [n− 2, 3], [n− 2, 2, 1] [n− 2, 1, 1] [n− 1, 1, 1], [n− 2, 2, 1], [n− 2, 1, 1, 1] [n− 3, 3] [n− 2, 3], [n− 3, 4], [n− 3, 3, 1] [1n] [2, 1n−1], [1n+1] [2, 1n−2] [3, 1n−2], [2, 2, 1n−3], [2, 1n−1] [2, 2, 1n−4] [3, 2, 1n−4], [2, 2, 2, 1n−5], [2, 2, 1n−3] [3, 1n−3] [4, 1n−3], [3, 2, 1n−4], [3, 1n−2] [2, 2, 2, 1n−6] [3, 2, 2, 1n−6], [2, 2, 2, 2, 1n−7], [2, 2, 2, 1n−5] Table 1: Constituents of χ, if res (χ)Sym(n) has a constituent with degree less than( n 3 ) − ( n 2 ) . By Frobenius reciprocity, for any character ϕ of Sym(n) ⟨res (χ)Sym(n), ϕ⟩Sym(n) = ⟨χ, ind (ϕ) Sym(n+1)⟩Sym(n+1). K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 537 Character Degree [n− 3, 4] (n+ 1)n(n− 1)(n− 7)/24 [n− 3, 3, 1] (n+ 1)n(n− 2)(n− 5)/8 [n− 2, 2, 1] (n+ 1)(n− 1)(n− 3)/3 [n− 2, 1, 1, 1] n(n− 1)(n− 2)/6 [2, 2, 2, 2, 1n−8] (n+ 1)n(n− 1)(n− 7)/24 [3, 2, 2, 1n−6] (n+ 1)n(n− 2)(n− 5)/8 [3, 2, 1n−4] (n+ 1)(n− 1)(n− 3)/3 [4, 1n−3] n(n− 1)(n− 2)/6 Table 2: Degrees of the characters from Table 1 that are larger than (n+1)n(n−4)6 for n ≥ 13. This means if ϕ is a constituent of res (χ)Sym(n), then χ is one of the constituents of ind (ϕ) Sym(n+1). The possible constituents of ind (ϕ)Sym(n+1) are recorded in Table 1; the second column lists the irreducible characters that, according to the branching rule, are constituents of representation of Sym(n+ 1) induced by the character in the first column. From these lists, and the degrees of the characters given in Table 2, we see that either χ is one of the ten listed in the theorem, or the degree of χ is larger than ( n 3 ) − ( n 2 ) (again, the degrees are calculated using the hook length formula). Thus res (χ)Sym(n) does not contain any of the ten irreducible characters of Sym(n) in the statement of the theorem. Next consider the case where the decomposition of res (χ)Sym(n) contains at least two irreducible characters of Sym(n) which are not in the list of the ten irreducible characters with dimension less ( n 3 ) − ( n 2 ) = n(n− 1)(n− 5)/6. In this case, the degree of χ must be at least n(n− 1)(n− 5)/3. But since n > 7, this is strictly larger than (n+1)n(n− 4)/6. Finally we need to consider the case where res (χ)Sym(n) contains exactly one irre- ducible character of Sym(n), which is not one of the ten listed in the theorem. By the branching rule the only irreducible characters of Sym(n+ 1) for which res (χ)Sym(n) con- tains only one irreducible character have a rectangular Young diagram, so χ = χ[st] for some s and t. Next consider res (χ)Sym(n−1), this is the restriction of χ = χ[st] to Sym(n− 1). By the branching rule, this can contain only the irreducible characters of n−1 that correspond to the partitions λ′ = [st−1, s− 2] and λ′′ = [st−2, s− 1, s− 1]. If λ′ is one of the ten partitions that correspond to irreducible characters of Sym(n− 1) with degree less than ( n−1 3 ) − ( n−1 2 ) , then one of the following cases must hold: • t = 1 and λ′ = [n− 1] and s = n+ 1, • t = 2 and λ′ = [n− 1, 1], [n− 2, 2] or [n− 3, 3], and s ≤ 5, or • 2 < t < 4 and and s ≤ 2. The first of these cases implies χ = [n + 1], which contradicts the degree of χ, and none of the other cases can happen, since n = st and n is assumed to be at least 13. Similarly, assume λ′′ = [st−2, s−1, s−1] is one of the partitions corresponding to the ten characters of Sym(n− 1) that have degree less than ( n−1 3 ) − ( n−1 2 ) . Then one of the following cases must hold: 538 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 • t = 2 and λ′′ = [s− 1, s− 1] and s ≤ 4, • 2 < t ≤ 5 and λ′′ = [st−1, 1, 1] and s ≤ 2, or • s = 1. The first two cases imply that n ≤ 10 and the final case implies that χ = [1(n+1)] which has degree 1. Thus res (χ)Sym(n−1) has two characters with degree at least ( n−1 3 ) − ( n−1 2 ) , so the degree of χ is at least (n− 1)(n− 2)(n− 6)/3, which is strictly greater than (n+1)n(n− 4)/6 for n ≥ 13. This is a contradiction, so no such χ exists. Next we will show that there are only three irreducible characters in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) that have degree no more than ( kℓ 3 ) − ( kℓ 2 ) . To do this we will consider the action of different Young subgroups on Uk,ℓ. For any integer partition λ ⊢ n we will denote the Young subgroup by Sym(λ) = Sym(λ1)× Sym(λ2)× · · · × Sym(λk). Theorem 3.4. Assume kℓ ≥ 13 and k ≥ 3. Then the only partitions in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) with dimension less than or equal to ( kℓ 3 ) − ( kℓ 2 ) are χ[kℓ], χ[kℓ−2,2], χ[kℓ−3,3]. Proof. Lemma 3.3 lists the 10 irreducible representations of Sym(kℓ) with dimension no more than ( kℓ 3 ) − ( kℓ 2 ) . We only need to show which of these representations are in the decomposition. The tool we use is Frobenius reciprocity along with the action of different Young subgroups on Uk,ℓ. By Frobenius reciprocity〈 ind (1Sym(λ)) Sym(kℓ) , ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) 〉 Sym(kℓ) =〈 1, res (ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) ) Sym(λ) 〉 Sym(λ) . The second inner product above gives the number of orbits of the action of Sym(λ) on the cosets Sym(kℓ)/(Sym(k) ≀ Sym(ℓ)); or, equivalently, the number of orbits of Sym(λ) on the partitions in Uk,ℓ. Using this fact with different Young subgroups will allow us to deter- mine that many of the representations with small degree do not occur in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ). To start, it is clear that Sym(kℓ) has one orbit on the (k, ℓ)-partitions, so χ[kℓ] has multiplicity 1 in the decomposition. Next consider the group Sym([kℓ− 1, 1]), it is also straight-forward that this group only has one orbit on the partitions. Using the definition of the Specht modules and the labelling of the irreducible characters of the symmetric group it is straight forward to see that ind (1Sym([kℓ−1,1])) Sym(kℓ) = χ[kℓ] + χ[kℓ−1,1], so we have that 〈 χ[kℓ] + χ[kℓ−1,1], ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) 〉 = 1. Since we know that χ[kℓ] occurs in this decomposition with multiplicity 1, this implies that χ[kℓ−1,1] does not occur in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ). K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 539 Next we consider the group Sym([kℓ− 2, 2]). This group has two orbits on the par- titions of Uk,ℓ. Again, from the definition of the Specht modules and the labelling of the irreducible characters, we have that〈 ind (1Sym([kℓ−2,2])) Sym(kℓ) , ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) 〉 =〈 χ[kℓ] + χ[kℓ−1,1] + χ[kℓ−2,2], ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) 〉 = 2. This implies that χ[kℓ−2,2] occurs in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) with multiplicity 1. We continue this process with the group Sym([kℓ− 2, 1, 1]). It has two orbits on the partitions of Uk,ℓ. Since ind (1Sym([kℓ−2,1,1])) Sym(kℓ) = χ[kℓ] + χ[kℓ−1,1] + χ[kℓ−2,2] + χ[kℓ−2,1,1], we conclude that χ[kℓ−2,1,1] does not occur in the decomposition. Next, we consider the group Alt(kℓ) ∩ Sym([kℓ− 2, 2]). This group has two orbits on the partitions of Uk,ℓ. Again the decomposition of ind (1Alt(kℓ)∩Sym([kℓ−2,2])) Sym(kℓ) is well-known (a proof can be found in [11, Proposition 1.4]) and we have ind (1Alt(kℓ)∩Sym([kℓ−2,2])) Sym(kℓ) = χ[kℓ] + χ[kℓ−1,1] + χ[kℓ−2,2] + χ[kℓ−2,1,1] + χ[1kℓ] + χ[2,1kℓ−2] + χ[2,2,1kℓ−4] + χ[3,1kℓ−3]. This implies that none of χ[1kℓ], χ[2,1kℓ−2], χ[2,2,1kℓ−4] and χ[3,1kℓ−3] occur in the decom- position of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ). Next we consider the group Sym([kℓ− 3, 3]). This group has three orbits on the parti- tions of Uk,ℓ and from the decomposition of ind (1Sym([kℓ−3,3])) Sym(kℓ) we have that ⟨ind (1Sym([kℓ−3,3])) Sym(kℓ) , ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ)⟩ = ⟨χ[kℓ] + χ[kℓ−1,1] + χ[kℓ−2,2] + χ[kℓ−3,3], ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ)⟩ = 3. This implies that χ[kℓ−3,3] occurs in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) with multiplicity 1. Next we consider the group Alt(kℓ) ∩ Sym([kℓ− 2, 1, 1]). This group has 2 orbits on the partitions of Uk,ℓ and ind (1Alt(kℓ)∩Sym([kℓ−2,1,1])) Sym(kℓ) = χ[kℓ] + χ[kℓ−1,1] + χ[kℓ−2,2] + χ[kℓ−2,1,1] χ[1kℓ] + χ[2,1kℓ−2] + χ[2,2,1kℓ−4] + χ[3,1kℓ−3]. Since χ[kℓ], and χ[kℓ−2,2] are in the decomposition, none of the irreducible representations χ[1kℓ], χ[2,1kℓ−2], χ[2,2,1kℓ−4], or χ[3,1kℓ−3] occur in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ). Finally, we consider the group Alt(kℓ)∩ Sym([kℓ− 3, 3]). This group has three orbits on the partitions of Uk,ℓ and ind (1Alt(kℓ)∩Sym([kℓ−3,3])) Sym(kℓ) = χ[kℓ] + χ[kℓ−1,1] + χ[kℓ−2,2] + χ[kℓ−3,3] + χ[1kℓ] + χ[2,1kℓ−2] + χ[22,1kℓ−4] + χ[23,1kℓ−6]. Which shows χ[2,2,2,1kℓ−6] is not in the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ). 540 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 4 Eigenvalues of Xk,ℓ with k ≥ 3 In this section we will find three of the eigenvalues of Xk,ℓ. For ease of notation, we will denote the irreducible representation of χλ by the λ-module. Also, the number of vertices in Xk,ℓ, which is equal to uk,ℓ, will be denoted simply by v and the degree of the graph Xk,ℓ will be simply written as d, rather than dk,ℓ. Any subgroup H ≤ Sym(kℓ) acts on the vertices of Xk,ℓ and the orbits of this action form an equitable partition. From any equitable partition, we can form a quotient graph and the eigenvalues of this quotient graph will be eigenvalues of the Xk,ℓ (details can be found in [13, Section 2.2]). The trivial case is H = Sym(kℓ), since this group is transitive, the equitable partition has all the vertices of Xk,ℓ in a single part. The quotient graph for this is simply the 1 × 1 matrix with the single entry d. The eigenvalue of this matrix is simply d, and the eigenvector is the all ones vector and the eigenspace is isomorphic to the trivial representation of Sym(kℓ). So d belongs to the [kℓ]-module. Since the subgroup Sym([kℓ− 1, 1]) has only one orbit on the vertices of Xk,ℓ, the next subgroup we consider is the Young subgroup Sym([kℓ− 2, 2]), considered as the stabilizer of the set {1, 2}. The action of Sym([kℓ− 2, 2]) on the partitions will give us another eigenvalue of the graph. Lemma 4.1. For integers k and ℓ, with k, ℓ ≥ 2, τ = − (k−1)dk(ℓ−1) is an eigenvalue of Xk,ℓ with multiplicity at least ( kℓ 2 ) − ( kℓ 1 ) . Proof. The action of Sym([kℓ− 2, 2]) on the (k, ℓ)-partitions has exactly 2 orbits: S1, the set of all partitions that have 1 and 2 in the same block, and S2, the set of all partitions in which 1 and 2 are in different blocks. The orbit S1 is a coclique in Xk,ℓ so the quotient matrix for this partition has the form( 0 d −τ d+ τ ) . (4.1) The eigenvalues of the quotient matrix (4.1) are d and τ . We can calculate the value of τ by counting edges between S1 and S2. Since S1 is a coclique, each vertex in it is adjacent to exactly d vertices in S2, and each vertex in S2 is adjacent to −τ vertices in S1. Using the sizes of S1 and S2, we have that the number of edges between S1 and S2 is equal to |S1| d = ( kℓ− 2 k − 2 ) uk,ℓ−1 d and also to |S2|(−τ) = ( kℓ− 2 k − 1 )( kℓ− k − 1 k − 1 ) uk,ℓ−2(−τ). Thus τ = − (k − 1)d k(ℓ− 1) (4.2) is a second eigenvalue for Xk,ℓ. Since this eigenvalue arises from the action of Sym([kℓ− 2, 2]), it belongs to a module that is common between the two representations ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) and ind (1Sym([kℓ−2,2])) Sym(kℓ). Thus it belongs to the mod- ule [kℓ − 2, 2], as this is the only common module, and must have dimension at least( kℓ 2 ) − ( kℓ 1 ) . K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 541 We denote this eigenvalue by τ since in Section 6 it will be shown that τ is the least eigenvalue of Xk,ℓ, provided that ℓ is sufficiently large. We also note that a second irre- ducible module may also have τ as the eigenvalue belonging to it, so the multiplicity of τ could be higher than the degree of the [kℓ− 2, 2]-module. Next we will consider the Young subgroup Sym([kℓ− 3, 3]), thought of as the group that stabilizes the set {1, 2, 3}. The action of this subgroup on Uk,ℓ has 3 orbits: T1, the set of all partitions with 1, 2, 3 in the same block; T2 the set of all partitions in which 1, 2, 3 are in exactly two different blocks; and T3 the set of all partitions in which 1, 2, 3 are in three different blocks. Any vertex in T1 is adjacent only to vertices in T3. Similarly, a vertex in T2 can be adjacent to vertices in T2 and T3. The quotient graph for this equitable partition is M = 0 0 d0 a d− a b c d− b− c  , where a, b, c are all non-negative. The eigenvalues for this quotient graph will be the eigenvalues that belong to modules that are both in the decomposition of ind (1Sym([kℓ−3,3])) Sym(kℓ) and the decomposition of ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ). Thus the eigenvalues will belong to the [kℓ], [kℓ − 2, 2] and [kℓ − 3, 3] modules. We have already seen that the eigenvalue for [kℓ] is d, and the eigenvalue for [kℓ− 2, 2] is τ . We will denote the eigenvalue belonging to [kℓ− 3, 3] by θ. Since the trace of the matrix is the sum of the eigenvalues we have that d+ a− b− c = d+ τ + θ. (4.3) The number of edges between T1 and T3 is equal to d|T1| = d ( kℓ− 3 k − 3 ) uk,ℓ−1, and also to b|T3| = b ( kℓ− 3 k − 1 )( kℓ− k − 2 k − 1 )( kℓ− 2k − 1 k − 1 ) uk,ℓ−3. Setting these equations equal to each other, then expanding the binomial coefficients and rearranging yields (k − 1)(k − 2) k2(ℓ− 1)(ℓ− 2) d = b. Replacing d = −k(ℓ−1)k−1 τ shows that b = − (k − 1)(k − 2) k2(ℓ− 1)(ℓ− 2) k(ℓ− 1) (k − 1) τ = − k − 2 k(ℓ− 2) τ. (4.4) Putting this into Equation 4.3 produces the following formula θ = a+ k − 2 k(ℓ− 2) τ − c− τ = a− c+ (k − 2)− k(ℓ− 2) k(ℓ− 2) τ. (4.5) 542 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 Similarly, counting the number of edges between T2 and T3 yields 3 ( kℓ− 3 k − 2 )( kℓ− k − 1 k − 1 ) uk,ℓ−2(d− a) =( kℓ− 3 k − 1 )( kℓ− k − 2 k − 1 )( kℓ− 2k − 1 k − 1 ) uk,ℓ−3(c). Again, expanding the binomial coefficients and rearranging shows that a = d− (ℓ− 2)k 3(k − 1) c. The characteristic polynomial of M is x3 + (−a+ b+ c− d)x2 + (−ab+ ad− bd− cd)x+ abd. Substituting in the values we have computed for b and c, and using the fact that τ is a root of the characteristic polynomial we get a = 2(k − 1) k(ℓ− 1) d. (4.6) From this we can compute that c = 3(kℓ− 3k + 2)(k − 1) k2(ℓ− 1)(ℓ− 2) d. (4.7) Lemma 4.2. For integers k and ℓ, with k, ℓ ≥ 3, θ = 2(k − 1)(k − 2)d k2(ℓ− 1)(ℓ− 2) is an eigenvalue of Xk,ℓ with multiplicity at least ( kℓ 3 ) − ( kℓ 2 ) . Proof. By Equations (4.5), (4.6) and (4.7), we can calculate that θ = 2(k − 1)(k − 2)d k2(ℓ− 1)(ℓ− 2) . (4.8) From the comments above, θ = 2(k−1)(k−2)dk2(ℓ−1)(ℓ−2) is the eigenvalue belonging to the unique [kℓ − 3, 3]-module in ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ). Since the dimension of the irreducible representation of [kℓ− 3, 3] is ( kℓ 3 ) − ( kℓ 2 ) , the multiplicity of θ is at least ( kℓ 3 ) − ( kℓ 2 ) . 5 Bound on degree of Xk,ℓ In this section we will find a lower bound on the degree of Xk,ℓ for all sufficiently large ℓ. If P and Q are two partitions that are adjacent in Xk,ℓ, then the meet table of P and Q is an ℓ × ℓ matrix with entries either 0 or 1, and further, the entries in each row and column in the meet table sum to k. We define Mk,ℓ to be the set of all such meet tables, so all ℓ× ℓ matrices with entries either 0 or 1, and row and columns sums equal to k. To find the K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 543 degree of Xk,ℓ, we first state a result on the number of such meet tables. Next, for a fixed partition P and a meet table M ∈ Mk,ℓ, we count the number of partitions Q for which the meet table of P and Q is M . Bender [1] determined the asymptotic cardinality of Mk,ℓ. (In fact, Bender found a much more general result, but we only state the result that we need here.) Theorem 5.1 (Bender [1]). For positive integers k, ℓ lim ℓ→∞ (k!)2ℓ (kℓ)! |Mk,ℓ| = e− (k−1)2 2 . To get a lower bound on dk,ℓ, we fix a partition P in Uk,ℓ, then for each M ∈ Mk,ℓ, we will count the number of Q so that the meet table of P and Q is M , then we use Theorem 5.1 to bound the size of Mk,ℓ. Lemma 5.2. For positive integers k, ℓ with k ≤ ℓ, dk,ℓ = k!ℓ ℓ! |Mk,ℓ|. Proof. Fix a partition P ∈ Uk,ℓ. Define a bipartite multigraph with the vertices in one part the meet tables in Mk,ℓ, and the vertices in the other part the partitions in the neighbour- hood of P in Xk,ℓ. Two vertices M and Q are adjacent if the meet table of P and Q is M . By counting the number of edges in this graph in two ways, we will determine the size of the neighbourhood of P in terms of |Mk,ℓ|. For any M ∈ Mk,ℓ, with M = [mi,j ] assume that row i corresponds to the block Pi ∈ P . Construct a partition Q = {Q1, Q2, . . . , Qℓ} so that the block Qj corresponds to column j of M and |Pi∩Qj | = mi,j . Since the entries of a row in M are either 0 or 1, and sum to k, there are k! ways to select how the elements from Pi will be distributed to the blocks of Q. So for each meet table M , there are k!ℓ partitions Q that can be constructed this way. It is possible that some of these partitions are equal, once the blocks are reordered, so this is a multigraph. For every Q in the neighbourhood of P , there are ℓ! ways to order the blocks of Q, once the blocks are ordered the meet table for P and Q is uniquely defined. In the bipartite graph, Q is adjacent to each of these tables in the graph (again, these tables may not be distinct, so the graph is a multigraph). The degree of every vertex Q is ℓ!. Thus we have that the number of edges in the multigraph is ℓ!dk,ℓ = ∑ M∈Mk,ℓ k!ℓ, and the result follows. Using Theorem 5.1 we have the asymptotic size of dk,ℓ. Corollary 5.3. For a fixed integer k with k ≥ 2, lim ℓ→∞ uk,ℓ dk,ℓ = e (k−1)2 2 . 544 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 Proof. From Equation (1.1) and Lemma 5.2, uk,ℓ dk,ℓ = (kℓ)! (k!)ℓℓ! ℓ! k!ℓ|Mk,ℓ| = (kℓ)! (k!)2ℓ|Mk,ℓ| . The result then follows from Theorem 5.1. Thus for every ϵ > 0, there exists an ℓ′ such that for all ℓ ≥ ℓ′, uk,ℓ dk,ℓ ≤ e (k−1)2 2 + ϵ. 6 A bound on the multiplicity of eigenvalues with large absolute value In Section 4 we found three eigenvalues, d, τ , and θ of Xk,ℓ, and the ratio between d and τ is dτ = k(1−ℓ) k−1 . If τ is the least eigenvalue of the graph, then by the ratio bound any coclique will have size no more than |V (Xk,ℓ)| 1− dτ = |V (Xk,ℓ)| 1− k(1−ℓ)k−1 = uk,ℓ−1. This is exactly the size of a set of canonically 2-intersecting (k, ℓ)-partitions. Thus our goal in this section is to show that τ is the least eigenvalue of Xk,ℓ. To this end, we first show if Xk,ℓ has an eigenvalue λ with λ2 > τ2, then there is a bound on the multiplicity of λ. Let {d(1), τ (mτ ), θ(mθ), λ(m2)2 , . . . , λ (mj) j } be the spectrum of the matrix Xk,ℓ, where the values mi represent the multiplicities of the eigenvalues. By squaring A and taking the trace, we have vd = d2 +mττ 2 +mθθ 2 + j∑ i=2 miλ 2 i . Hence for every 2 ≤ i ≤ j we have vd− d2 −mττ2 −mθθ2 ≥ miλ2i . Assume λi is an eigenvalue of Xk,ℓ with λ2i > τ 2, and also that λi is not the eigenvalue belonging to the [kℓ], [kℓ− 2, 2] or [kℓ− 3, 3] modules, then vd− d2 −mττ2 −mθθ2 τ2 ≥ mi. Expanding θ using Equation (4.8) in the above equation produces the following equation(v d − 1 ) k2(ℓ− 1)2 (k − 1)2 −mθ 4(k − 2)2 k2(ℓ− 2)2 −mτ ≥ mi. Further, by Lemmas 4.1 and 4.2, it is known that mτ ≥ ( kℓ 2 ) − ( kℓ 1 ) and mθ ≥ ( kℓ 3 ) − ( kℓ 2 ) , so this bound becomes(v d − 1 ) k2(ℓ− 1)2 (k − 1)2 − (kℓ)(kℓ− 1)(kℓ− 5) 6 4(k − 2)2 k2(ℓ− 2)2 − (kℓ)(kℓ− 3) 2 ≥ mi. K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 545 Our next step is to show that this upper bound on mi is smaller than ( kℓ 3 ) − ( kℓ 2 ) . This will be a contradiction with Theorem 3.4 since we have assumed that λ does not belong to any of the [kℓ], [kℓ− 2, 2], and [kℓ− 3, 3] modules. In other words, we need to prove that v d − 1 < ℓ(k − 1)2 ( k2(ℓ− 2)2(kℓ− 4)(kℓ+ 1) + 4(k − 2)2(kℓ− 1)(kℓ− 5) ) 6k3(ℓ− 1)2(ℓ− 2)2 . (6.1) In the next result, we will see that this will follow from Corollary 5.3. Theorem 6.1. Fix an integer k ≥ 3. For ℓ sufficiently large, the largest set of partially 2-intersecting uniform (k, ℓ)-partitions has size( kℓ− 2 k − 2 ) uk,ℓ−1. Proof. For any distinct i, j ∈ {1, . . . , kℓ}, the set Si,j of all (k, ℓ)-partitions with i and j in the same block form a set of partially 2-intersecting (k, ℓ)-partitions of the size given in the theorem. Corollary 5.3 shows that vd approaches a fixed constant, namely e (k−1)2 2 , as ℓ goes to infinity. Since the right hand side of Equation (6.1) grows linearly in ℓ, we have that Equation (6.1) holds for ℓ sufficiently large. This implies if there is an eigenvalue λ of Xk,ℓ with λ ≤ τ , then the multiplicity of λ is less than or equal to ( kℓ 3 ) − ( kℓ 2 ) . By Theorem 3.4, eigenspaces with dimension less than or equal to ( kℓ 3 ) − ( kℓ 2 ) can only include the [kℓ], [kℓ− 2, 2] or the [kℓ− 3, 3]-modules. The degree, d, is the eigenvalue be- longing to the [kℓ]-module, and Lemma 4.1 and Lemma 4.2 shows that τ is the eigenvalue belonging to the [kℓ − 2, 2]-module and θ belongs to the [kℓ − 3, 3]-module. So we can conclude that τ = − (k−1)dk(ℓ−1) is the least eigenvalue of Xk,ℓ and that τ belongs only to the [kℓ− 2, 2]-module. By the ratio bound, Theorem 2.1, the maximum size of coclique in Xk,ℓ is |V (Xk,ℓ)| 1− dτ = v 1− d − (k−1)d k(ℓ−1) = v 1 + k(ℓ−1)k−1 = v(k − 1) kℓ− 1 = ( kℓ− 2 k − 2 ) uk,ℓ−1. The previous result shows that the sets Si,j are the largest intersecting sets. We further conjecture that these sets are the only maximum intersecting sets. Conjecture 6.2. For k ≥ 3 and ℓ sufficiently large, the only sets of partially 2-intersecting (k, ℓ)-partitions with size ( kℓ−2 k−2 ) uk,ℓ−1 are the sets Si,j . We can make a step towards this conjecture with the following weaker characterization of the maximum intersecting sets. Denote the characteristic vectors of the sets Si,j by vi,j . Corollary 6.3. For a fixed integer k ≥ 3 and ℓ sufficiently large, let S be any maximum partially 2-intersecting set of (k, ℓ)-partitions. Then the characteristic vector of S is a linear combination of the vectors vi,j . Proof. For k ≥ 3 and ℓ sufficiently large, Si,j is a maximum coclique in Xk,ℓ and equality holds in the ratio bound. Let vi,j be the characteristic vector of Si,j . Since we have equality in the ratio bound, this implies that vi,j − k − 1 kℓ− 1 1 546 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 is a τ -eigenvector (where 1 is the all ones vector). Since no other modules have have eigenvalue τ , these vectors are in the [kℓ− 2, 2]-module. Further, the set of vectors{ vi,j − k − 1 kℓ− 1 1 | i, j ∈ {1, . . . , kℓ} } is invariant under the action of Sym(kℓ), so they form a module. Since the [kℓ − 2, 2]- module is irreducible, these vectors span the entire [kℓ − 2, 2]-module; this also implies that the vectors {vi,j | i, j ∈ {1, . . . , kℓ}} span the [kℓ] and [kℓ− 2, 2]-modules. Let S be a partially 2-intersecting set of (k, ℓ)-partition of maximum size, and let vS denote the characteristic vector of S. Then vS − k−1kℓ−1 1 is in the [kℓ− 2, 2]-module. Thus vS is in the span of the [kℓ] and [kℓ − 2, 2]-module, so vS is a linear combination of the vectors vi,j . 7 Exact result for k = 3 In this section we will prove Theorem 6.1 holds for all ℓ ≥ 3, when k = 3. It is already known, see Corollary 7.5.6 in [19], that Theorem 6.1 holds in the case where k = 3 and ℓ is odd; this follows from the existence of resolvable packing designs of an appropriate size. For k = 3, we observed experimentally that the ratio u3,ℓ/d3,ℓ converges to e (k−1)2 2 = e2 surprisingly quickly. If the sequence of u3,ℓ/d3,ℓ was non-increasing this would be sufficient, but we have no proof of this. Rather, in this section we show an upper bound on the ratio u3,ℓ/d3,ℓ for all ℓ, or, equivalently, a lower bound on d3,ℓ. This bound holds for ℓ > 10, and we simply directly check the theorem for the specific graphs with smaller values of ℓ. Lemma 7.1. For ℓ > 10, the degree, d3,ℓ is greater than u3,ℓ/24. Proof. We will use a truncated inclusion-exclusion argument to bound the degree. Since Xk,ℓ is vertex transitive, we obtain a bound on the degree by counting the neighbours of an arbitrary partition P ∈ U3,ℓ. Fix a partition P ∈ U3,ℓ and let J be the set of pairs {x, y} of elements in {1, 2, . . . , 3ℓ} that are contained in the same block of P . Note that |J | = 3ℓ. For a pair {x, y} ∈ J , we define A{x,y} to be the set of all partitions which contain x and y in the same block. Further, for a subset J ⊆ J , define N(J) = |∩{x,y}∈JA{x,y}| and for 0 ≤ j ≤ 3ℓ let Nj = ∑ J,|J|=j N(J). By inclusion-exclusion, d3,ℓ = 3ℓ∑ j=0 −1jNj . (7.1) Next we calculate Nj . First, we note that N0 = N(∅) = u3,ℓ. For any subset J ⊆ J , each block in P contains either 0, 1, 2 or 3 of the pairs from J . For i = 0, 1, 2, 3, let ni be the number of blocks in P that have exactly i of their pairs in J . We call the 4-tuple (n0, n1, n2, n3) the pair distribution of J and note that n0 + n1 + n2 + n3 = ℓ. For each block of P its block type relative to J is the number of pairs in J that are in the block. With this terminology, we can find Nj . K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 547 First, fix a subset J ⊆ J with pair distribution (n0, n1, n2, n3) and count the number of partitions Q ∈ ∩{x,y}∈JA{x,y}. Each block of P with block type n3 relative to J determines exactly which three elements are in a block of Q, as do the blocks of type n2. Each of the blocks of type n1 determines two of the three points in the block of Q. One more point must be chosen to complete each of these blocks, and this choice is ordered since each pair of type n1 from J uniquely labels its corresponding block. Each of the blocks of type n0 does not determine any points in Q. Thus the number of partitions Q which contain the pairs from J is given by the multinomial coefficient 1 n0! ( 3ℓ− 3(n3 + n2)− 2n1 1(n1), 3(n0) ) (where the exponent in braces indicates the number is repeated that many times). We now count the number of possible J which have pair distribution (n0, n1, n2, n3). The number of ways to select the type of each block in P is equal to the multinomial coefficient ( ℓ n0, n1, n2, n3 ) , since we are choosing the blocks from P that have either 0, 1, 2 or 3 pairs in J . Each of the blocks of P with type n3 has all of its three pairs in J , while for each block of type n2 there are three ways to choose which two of the three possible pairs are in J . Similarly, for each block of type n1 there is one pair in J , and there are three ways to chose this pair. Finally, each of the n0 blocks does not contribute any pairs to J . Thus there are 3n1+n2 different sets J in with pair distribution (n0, n1, n2, n3). Finally, we sum the number of partitions Q ∈ ∩{x,y}∈JA{x,y} over all possible pair distributions that J can have. Each pair distribution (n0, n1, n2, n3) is an ordered partition of ℓ into exactly four non-negative parts. The pair distribution (n0, n1, n2, n3) corresponds to a set J of size n1+2n2+3n3. Define C(ℓ, j) to be the set of compositions of ℓ into four parts with n1 + 2n2 + 3n3 = j. Then from our previous counting we have that Nj = ∑ (n0,n1,n2,n3)∈C(ℓ,j) 3n1+n2 ( ℓ n0, n1, n2, n3 ) 1 n0! ( 3ℓ− 3(n3 + n2)− 2n1 1(n1), 3(n0) ) . When we put this value in Equation 7.1 and truncate this sum after an odd j we will get a lower bound on d3,ℓ. Taking j up to 5 we sum over the following list of pair distributions: C(ℓ, 0) = {(ℓ, 0, 0, 0)} C(ℓ, 1) = {(ℓ− 1, 1, 0, 0)} C(ℓ, 2) = {(ℓ− 1, 0, 1, 0), (ℓ− 2, 2, 0, 0)} C(ℓ, 3) = {(ℓ− 1, 0, 0, 1), (ℓ− 2, 1, 1, 0), (ℓ− 3, 3, 0, 0)} C(ℓ, 4) = {(ℓ− 2, 1, 0, 1), (ℓ− 2, 0, 2, 0), (ℓ− 3, 2, 1, 0), (ℓ− 4, 4, 0, 0)} C(ℓ, 5) = {(ℓ− 2, 0, 1, 1), (ℓ− 3, 2, 0, 1), (ℓ− 3, 1, 2, 0), (ℓ− 4, 3, 1, 0), (ℓ− 5, 5, 0, 0)} 548 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 Expanding this becomes d3,ℓ ≥ 5∑ j=0 −1j ∑ (n0,n1,n2,n3)∈C(ℓ,j) 3n1+n2 ( ℓ n0, n1, n2, n3 )(3ℓ−3(n3+n2)−2n1 1(n1),3(n0) ) n0! = ( ℓ ℓ )( 3ℓ 3(ℓ) ) (ℓ)! + −3 ( ℓ ℓ−1,1 )( 3ℓ−2 1,3(ℓ−1) ) (ℓ− 1)! + 3 ( ℓ ℓ−1,1 )( 3ℓ−3 3(ℓ−1) ) (ℓ− 1)! + 32 ( ℓ ℓ−2,2 )( 3ℓ−4 1(2),3(ℓ−2) ) (ℓ− 2)! + − ( ℓ ℓ−1,1 )( 3ℓ−3 3(ℓ−1) ) (ℓ− 1)! + −32 ( ℓ ℓ−2,1(2) )( 3ℓ−5 1,3(ℓ−2) ) (ℓ− 2)! + −33 ( ℓ ℓ−3,3 )( 3ℓ−6 1(3),3(ℓ−3) ) (ℓ− 3)! + 3 ( ℓ ℓ−2,1(2) )( 3ℓ−5 1,3(ℓ−2) ) (ℓ− 2)! + 32 ( ℓ ℓ−2,2 )( 3ℓ−6 3(ℓ−2) ) (ℓ− 2)! + 33 ( ℓ ℓ−3,2,1 )( 3ℓ−7 1(2),3(ℓ−3) ) (ℓ− 3)! + 34 ( ℓ ℓ−4,4 )( 3ℓ−8 1(4),3(ℓ−4) ) (ℓ− 4)! + −3 ( ℓ ℓ−2,1(2) )( 3ℓ−6 3(ℓ−2) ) (ℓ− 2)! + −32 ( ℓ ℓ−3,2,1 )( 3ℓ−7 1(2),3(ℓ−3) ) (ℓ− 3)! + −33 ( ℓ ℓ−3,1,2 )( 3ℓ−8 1,3(ℓ−3) ) (ℓ− 3)! + −34 ( ℓ ℓ−4,3,1 )( 3ℓ−9 1(3),3(ℓ−4) ) (ℓ− 4)! + −35 ( ℓ ℓ−5,5 )( 3ℓ−10 1(5),3(ℓ−5) ) (ℓ− 5)! = (243ℓ6 − 2997ℓ5 + 13905ℓ4 − 32355ℓ3 + 42732ℓ2 − 32728ℓ+ 11200)(3ℓ− 10)! 80(6ℓ−4)(ℓ− 10)!(ℓ6 − 39ℓ5 + 625ℓ4 − 5265ℓ3 + 24574ℓ2 − 60216ℓ+ 60480) . Thus u3,ℓ d3,ℓ ≤ 5(729ℓ 6 − 6561ℓ5 + 23085ℓ4 − 40095ℓ3 + 35586ℓ2 − 14904ℓ+ 2240) 243ℓ6 − 2997ℓ5 + 13905ℓ4 − 32355ℓ3 + 4273ℓ2 − 32728ℓ+ 11200 . For ℓ > 10 this gives that u3,ℓ/d3,ℓ < 24. Theorem 7.2. For k = 3 and all ℓ ≥ 3 the largest set of partially 2-intersecting uniform partitions has size (3ℓ− 2)u3,ℓ−1. Proof. For ℓ = 3 all the eigenvalues of X3,3 have long been known to be {36, 8, 2, −4, −12} [18]. The ratio bound holds with equality, and the only irreducible representation that belongs to the least eigenvalue is χ[7,2]. For ℓ = 4, all the eigenvalues of X3,4 are {1296, 96, 72, 48, 32, 0,−24,−48,−288}. These can be calculated by making a quotient graph of X3,4 from the action of Sym(3) ≀ Sym(4) on the partitions. This equitable partition has a cell of size 1, so the eigenvalues of the quotient graph are exactly the eigenvalues of X3,4. Further, the multiplicities of the eigenvalues can be calculated using the formulas in [14, Section 5.3] and the [10, 2]-module is the only module to which the eigenvalue −288 belongs. For ℓ ∈ {5, . . . , 12} the only irreducible representations with dimension less then( 3ℓ 3 ) − ( 3ℓ 2 ) in the decomposition of ind (1Sym(3)≀Sym(ℓ)) Sym(3ℓ) are the three listed in Theo- rem 3.4—this can be checked using GAP [10]. Thus Theorem 3.4 holds for all 5 ≤ ℓ ≤ 12 when k = 3. For all ℓ > 10, Lemma 7.1 shows that uk,ℓ/dk,ℓ − 1 < 23. In this same range, the right hand side of Equation (6.1) is at least 26. Thus the inequality from Equation (6.1) holds for all ℓ > 10. K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 549 For 5 ≤ ℓ ≤ 10 the degrees d3,ℓ can be directly computed d3,5 = 132192, d3,7 = 3829057920, d3,9 = 333973115062272, d3,6 = 19258560, d3,8 = 1001695548672, d3,10 = 138348645213579264, and the inequality from Equation (6.1) directly checked. 8 Further work In this paper we only consider partially 2-intersecting partitions, but the conjecture in [20] is for partial t-intersection sets of partitions with k ≤ ℓ(t − 1). It is possible that the approach in this paper could be applied for larger values of t, but there are some steps that we predict will be complicated. It is straight-forward to generalize the definition of Xk,ℓ to partially t-intersecting par- titions by defining the graph Xt,k,ℓ. This graph will also have Uk,ℓ as its vertex set, and two partitions P and Q are adjacent if and only if for all pairs of blocks Pi ∈ P and Qj ∈ Q we have |Pi ∩Qj | < t. A partially t-intersecting set of partitions is a coclique in Xt,k,ℓ. The conjecture is if k < ℓ(t − 1), then the maximum cocliques in Xt,k,ℓ are exactly the canonical partially t-intersecting sets. The Young subgroup Sym([kℓ− t, t]) is the stabilizer of a canonically partially t-intersecting set. The most significant complication is that for t > 2, there are more than two irreducible representations in both ind (1Sym([kℓ−t,t])) Sym(kℓ) and ind (1Sym(k)≀Sym(ℓ)) Sym(kℓ) . (8.1) For the approach given in this paper to work, we believe the eigenvalues belonging to all the irreducible representations common to these two induced representations, except the trivial representation, should be the least eigenvalue of Xt,k,ℓ. To make this happen we suspect that a weighted adjacency matrix of Xt,k,ℓ would be needed in the ratio bound, rather than just the adjacency matrix; the weighting would have to be chosen so that the common modules (except the trivial) in the representations in (8.1) all belong to the same eigenvalue. Another complication is that potentially more of the eigenvalues of Xt,k,ℓ would have to be calculated, at the very least all the eigenvalues belonging to the common representations would need to be known. Bender’s theorem is much more general than the version we stated here. We only state Bender’s theorem for matrices with 01-entries, but the full theorem applies to matrices with entries less than t. Using the full theorem we would be able to approximate the degree of Xt,k,ℓ for t ≥ 2. ORCID iDs Karen Meagher https://orcid.org/0000-0002-7948-9149 Mahsa N. Shirazi https://orcid.org/0000-0003-3924-0848 Brett Stevens https://orcid.org/0000-0003-4336-1773 References [1] E. A. Bender, The asymptotic number of non-negative integer matrices with given row and col- umn sums, Discrete Math. 10 (1974), 217–223, doi:10.1016/0012-365X(74)90118-6, https: //doi.org/10.1016/0012-365X(74)90118-6. 550 Ars Math. Contemp. 23 (2023) #P4.02 / 531–551 [2] F. Brunk, Intersection Problems in Combinatorics, Ph.D. thesis, University of St. Andrews, 2009, http://hdl.handle.net/10023/765. [3] T. Ceccherini-Silberstein, F. Scarabotti and F. Tolli, Representation Theory of the Symmet- ric Groups: The Okounkov-Vershik Approach, Character Formulas, and Partition Algebras, Cambridge Studies in Advanced Mathematics, Cambridge University Press, Cambridge, 2010, doi:10.1017/cbo9781139192361, https://doi.org/10.1017/cbo9781139192361. [4] P. Delsarte, An algebraic approach to the association schemes of coding theory, Philips Res. Rep. Suppl. (1973), vi+97. [5] D. Ellis, Setwise intersecting families of permutations, J. Comb. Theory Ser. A 119 (2012), 825– 849, doi:10.1016/j.jcta.2011.12.003, https://doi.org/10.1016/j.jcta.2011. 12.003. [6] D. Ellis, E. Friedgut and H. Pilpel, Intersecting families of permutations, J. Am. Math. Soc. 24 (2011), 649–682, doi:10.1090/S0894-0347-2011-00690-5, https://doi.org/ 10.1090/S0894-0347-2011-00690-5. [7] P. Erdős, C. Ko and R. Rado, Intersection theorems for systems of finite sets, Quart. J. Math. Oxford Ser. (2) 12 (1961), 313–320, doi:10.1093/qmath/12.1.313, https://doi.org/10. 1093/qmath/12.1.313. [8] P. L. Erdős and L. A. Székely, Erdős-Ko-Rado theorems of higher order, in: Numbers, In- formation and Complexity (Bielefeld, 1998), Kluwer Acad. Publ., Boston, MA, pp. 117–124, 2000. [9] S. Fallat, K. Meagher and M. N. Shirazi, The Erdős-Ko-Rado theorem for 2-intersecting fam- ilies of perfect matchings, Algebr. Comb. 4 (2021), 575–598, doi:10.5802/alco.169, https: //doi.org/10.5802/alco.169. [10] The GAP Group, GAP – Groups, Algorithms, and Programming, Version 4.11.1, 2021, {https://www.gap-system.org}. [11] C. Godsil and K. Meagher, Multiplicity-free permutation representations of the symmetric group, Ann. Comb. 13 (2010), 463–490, doi:10.1007/s00026-009-0035-8, https://doi. org/10.1007/s00026-009-0035-8. [12] C. Godsil and K. Meagher, Erdős-Ko-Rado Theorems: Algebraic Approaches, volume 149 of Cambridge Studies in Advanced Mathematics, Cambridge University Press, Cambridge, 2016, doi:10.1017/cbo9781316414958, https://doi.org/10.1017/cbo9781316414958. [13] C. Godsil and K. Meagher, An algebraic proof of the Erdős-Ko-Rado theorem for intersect- ing families of perfect matchings, Ars Math. Contemp. 12 (2017), 205–217, doi:10.26493/ 1855-3974.976.c47, https://doi.org/10.26493/1855-3974.976.c47. [14] C. D. Godsil, Algebraic Combinatorics, Chapman and Hall Mathematics Series, Chapman & Hall, New York, 1993. [15] W. H. Haemers, Hoffman’s ratio bound, Linear Algebra Appl. 617 (2021), 215–219, doi:10. 1016/j.laa.2021.02.010, https://doi.org/10.1016/j.laa.2021.02.010. [16] N. Lindzey, Erdős-Ko-Rado for perfect matchings, European J. Comb. 65 (2017), 130–142, doi:10.1016/j.ejc.2017.05.005, https://doi.org/10.1016/j.ejc.2017.05.005. [17] L. Long, R. Plaza, P. Sin and Q. Xiang, Characterization of intersecting families of maximum size in PSL(2, q), J. Comb. Theory Ser. A 157 (2018), 461–499, doi:10.1016/j.jcta.2018.03. 006, https://doi.org/10.1016/j.jcta.2018.03.006. [18] R. Mathon and A. Rosa, A new strongly regular graph, J. Comb. Theory Ser. A 38 (1985), 84–86, doi:10.1016/0097-3165(85)90025-1, https://doi.org/10.1016/ 0097-3165(85)90025-1. K. Meagher et al.: An extension of the Erdős-Ko-Rado theorem to uniform set partitions 551 [19] K. Meagher, Covering arrays on graphs: qualitative independence graphs and extremal set partition theory, Ph.D. thesis, University of Ottawa, 2005, http://doi.org/10.20381/ ruor-19660. [20] K. Meagher and L. Moura, Erdős-Ko-Rado theorems for uniform set-partition systems, Elec- tron. J. Comb. 12 (2005), Research Paper 40, 12, doi:10.37236/1937, https://doi.org/ 10.37236/1937. [21] K. Meagher and A. S. Razafimahatratra, The Erdős-Ko-Rado theorem for 2-pointwise and 2-setwise intersecting permutations, Electron. J. Comb. 28 (2021), Paper No. 4.10, 21, doi: 10.37236/9556, https://doi.org/10.37236/9556. [22] K. Meagher and P. Spiga, An Erdős-Ko-Rado theorem for the derangement graph of PGL3(q) acting on the projective plane, SIAM J. Discrete Math. 28 (2014), 918–941, doi:10.1137/ 13094075x, https://doi.org/10.1137/13094075x. [23] K. Meagher, P. Spiga and P. H. Tiep, An Erdős-Ko-Rado theorem for finite 2-transitive groups, Eur. J. Comb. 55 (2016), 100–118, doi:10.1016/j.ejc.2016.02.005, https://doi.org/10. 1016/j.ejc.2016.02.005. [24] B. E. Sagan, The Symmetric Group: Representations, Combinatorial Algorithms, and Symmetric Functions, volume 203 of Graduate Texts in Mathematics, Springer-Verlag, New York, 2nd edition, 2001, doi:10.1007/978-1-4757-6804-6, https://doi.org/10. 1007/978-1-4757-6804-6. [25] P. Spiga, The Erdős-Ko-Rado theorem for the derangement graph of the projective general linear group acting on the projective space, J. Comb. Theory Ser. A 166 (2019), 59–90, doi: 10.1016/j.jcta.2019.02.015, https://doi.org/10.1016/j.jcta.2019.02.015. [26] R. M. Wilson, The exact bound in the Erdős-Ko-Rado theorem, Combinatorica 4 (1984), 247– 257, doi:10.1007/bf02579226, https://doi.org/10.1007/bf02579226. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.03 / 553–562 https://doi.org/10.26493/1855-3974.2683.5f3 (Also available at http://amc-journal.eu) Almost simple groups as flag-transitive automorphism groups of symmetric designs with λ prime* Seyed Hassan Alavi † , Ashraf Daneshkhah , Fatemeh Mouseli Department of Mathematics, Faculty of Science, Bu-Ali Sina University, Hamedan, Iran Received 19 August 2021, accepted 10 January 2023, published online 28 February 2023 Abstract In this article, we study symmetric designs with λ prime admitting a flag-transitive and point-primitive automorphism group G of almost simple type with socle X . We prove that either D is one of the six well-known examples of biplanes and triplanes, or D is the point- hyperplane design of PG(n−1, q) with λ = (qn−2−1)/(q−1) prime and X = PSLn(q). Keywords: Almost simple group, automorphism group, flag-transitive, point-primitive, symmetric de- sign. Math. Subj. Class. (2020): 05B05, 05B25, 20B25, 20D05 1 Introduction Symmetric designs admitting flag-transitive automorphism groups are of most interest. Kantor [14] classified flag-transitive symmetric (v, k, 1) designs known as projective planes of order n, and showed that either D is a Desarguesian projective plane and PSL3(n)⊴G, or G is a sharply flag-transitive Frobenius group of odd order (n2 + n+ 1)(n+ 1), where n is even and n2 + n + 1 is prime. Regueiro gave a classification of nontrivial symmetric designs with λ = 2 (biplanes) admitting flag-transitive automorphism groups apart from those groups contained in a 1-dimensional affine group [17, 18, 19, 20, 21]. Dong, Fang and Zhou studied flag-transitive automorphism groups G of nontrivial symmetric (v, k, 3) de- signs (triplanes), and in conclusion, excluding the case where G ⩽ AΓL1(q) where q = pm with p ⩾ 5 prime, they determined all such possible symmetric designs [13, 25, 26, 27, 28]. *The authors would like to thank anonymous referees for providing us helpful and constructive comments and suggestions. †Corresponding author. E-mail addresses: alavi.s.hassan@basu.ac.ir (Seyed Hassan Alavi), adanesh@basu.ac.ir (Ashraf Daneshkhah), f.mouseli@sci.basu.ac.ir (Fatemeh Mouseli) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 554 Ars Math. Contemp. 23 (2023) #P4.03 / 553–562 Table 1: Some symmetric designs with λ prime. Line v k λ X H G Designs References∗ 1 7 4 2 PSL2(7) S4 PSL2(7) Complement of the Fano plane [2, 8] 2 11 5 2 PSL2(11) A5 PSL2(11) Hadamard [2, 8] 3 11 6 3 PSL2(11) A5 PSL2(11) Complement of line 2 [2, 8] 4 15 7 3 A7 PSL3(2) A7 PG2(3, 2) [8, 26, 29] 5 45 12 3 PSU4(2) 2.(A4×A4).2 PSU4(2) - [7, 11, 22] 6 45 12 3 PSU4(2) 2.(A4×A4).2:2 PSU4(2):2 - [7, 11, 22] Note: The last column addresses to references in which a design with the parameters in the line has been constructed. Recently, Z. Zhang, Y. Zhang and Zhou in [24] proved that if D is a nontrivial symmet- ric (v, k, λ) design with λ prime and G is a flag-transitive and point-primitive automor- phism group of D, then G must be of affine or almost simple type. In this paper, we study symmetric designs with λ prime admitting a flag-transitive and point-primitive automor- phism group of almost simple type. Indeed, we have already shown in [4] that almost simple exceptional groups of Lie type give rise to no possible symmetric designs with λ prime. In addition, we investigated the case where G is an almost simple group with so- cle X being a finite simple classical group of Lie type, and proved that D is either the point-hyperplane design of a projective space PG(n− 1, q), or it is of parameters (7, 4, 2), (11, 5, 2), (11, 6, 3) or (45, 12, 3). Here, we focus on the case where G is an almost simple group with socle X an alternating group: Theorem 1.1. Let D be a nontrivial symmetric (v, k, λ) design with λ prime and α a point of D, and let G be a flag-transitive and point-primitive automorphism group of D of almost simple type. If the socle of G is an alternating group Ac with c ⩾ 5, then D = PG2(3, 2) with parameters (15, 7, 3) and G = A7 with the point-stabiliser Gα = PSL3(2). By Theorem 1.1 and the main results of [3, 23], we consequently obtain all symmetric designs with λ prime admitting flag-transitive and point-primitive automorphism groups of almost simple type: Corollary 1.2. Let D be a nontrivial symmetric (v, k, λ) design with λ prime. Suppose that G is a flag-transitive and point-primitive automorphism group of D of almost simple type with socle X . Then λ = 2, 3 and (D, G) is as in one of the lines of Table 1, or D is the point-hyperplane design of PG(n − 1, q) with λ = (qn−2 − 1)/(q − 1) prime and X = PSLn(q). In order to prove Theorem 1.1, for the case where λ ⩽ 100 or gcd(k, λ) = 1, by [29, 30], we obtain the designs in the statement. Then we assume that λ > 100 and λ divides k (as λ is prime). In this case, we show that there is no symmetric deign with λ prime and flag-transitive and point-primitive automorphism group G. Here, we first observe that the point-stabiliser H of G has to be large, that is to say, |G| ⩽ |H|3, see Corollary 2.2. The possibilities for H can be read off from [5]. In Section 3, we examine these possibilities and achieve our desired result. In Section 4, we give a detailed proof of Corollary 1.2 which follows immediately from Theorem 1.1 and the main results of [3, 23]. S. H. Alavi et al.: Almost simple groups as flag-transitive automorphism groups . . . 555 1.1 Definitions and notation All groups and incidence structures in this paper are finite. A group G is said to be almost simple with socle X if X ⊴ G ⩽ Aut(X), where X is a nonabelian simple group. Sym- metric and alternating groups on c letters are denoted by Sc and Ac, respectively. We write “n” for group of order n. A symmetric (v, k, λ) design D is a pair (P,B) with a set P of v points and a set B of v blocks such that each block is a k-subset of P and each pair of dis- tinct points is contained in exactly λ blocks. We say that D is nontrivial if 2 < k < v − 1. A flag of D is a point-block pair (α,B) such that α ∈ B. An automorphism of D is a permutation on P which maps blocks to blocks and preserving the incidence. The full au- tomorphism group Aut(D) of D is the group consisting of all automorphisms of D. For G ⩽ Aut(D), G is called flag-transitive if G acts transitively on the set of flags. The group G is said to be point-primitive if G acts primitively on P . For a given positive integer n and a prime divisor p of n, we denote the p-part of n by np, that is to say, np = pt with pt | n but pt+1 ∤ n. Further notation and definitions in both design theory and group theory are standard and can be found, for example in [6, 9, 12, 15, 16]. 2 Preliminaries In this section, we state some useful facts in both design theory and group theory. If a group G acts on a set P and α ∈ P , the subdegrees of G are the length of orbits of the action of the point-stabiliser Gα on P . Lemma 2.1 ([2, Lemma 2.1]). Let D be a symmetric (v, k, λ) design, and let G be a flag-transitive automorphism group of D. If α is a point of D, then (i) k(k − 1) = λ(v − 1); (ii) k divides |Gα|, and λv < k2; (iii) k | λd, for all nontrivial subdegrees d of G. For a point-stabiliser H of a flag-transitive automorphism group G of a design D, by Lemma 2.1(ii), we conclude that λ|G| ⩽ |H|3, and so we have that Corollary 2.2. Let D be a symmetric (v, k, λ) design with a flag-transitive automorphism group G and α a point of D. Then |G| ⩽ |Gα|3. Lemma 2.3. Suppose that s and t are positive integers. Then (i) if t > s ⩾ 9, then ( s+t s ) > s2t3; (ii) if s ⩾ 4 and there exists t0 ⩾ 7 such that ( s+t0 s ) > s2t30, then ( s+t s ) > s2t3 for all t ⩾ t0. Proof. (i) If t > s = 9, then we observe that the inequality ( s+t s ) = ( t+9 9 ) > 81t3 = s2t3 holds. If t > s ⩾ 10, then 10 ⩽ s ⩽ s+t2 , and so that ( s+t s ) ⩾ ( 10+t 10 ) > t5. Since t > s, we have that t5 > s2t3, and hence ( s+t s ) > t5 > s2t3. 556 Ars Math. Contemp. 23 (2023) #P4.03 / 553–562 (ii) It suffices to show that ( s+t0+1 s ) > s2(t0 + 1) 3. Note that( s+ t0 + 1 s ) = ( s+ t0 s ) (s+ t0 + 1) (t0 + 1) > s2t30 (s+ t0 + 1) (t0 + 1) = s2(t0 + 1) 3 (s+ t0 + 1)t 3 0 (t0 + 1)4 . Since t0 ⩾ 7 and s ⩾ 4, it follows that (s+ t0+1)t30 ⩾ (t0+5)t 3 0 > (t0+1) 4. Therefore,( s+t0+1 s ) > s2(t0 + 1) 3. 3 Proof of Theorem 1.1 Suppose that G is a flag-transitive and point-primitive automorphism group of a symmetric (v, k, λ) design D with λ prime. Suppose that X is the alternating group Ac of degree c ⩾ 5 on Ω = {1, . . . , c} and that H := Gα with α a point of D. Then H is maximal in G by [12, Corollary 1.5A], and since G = HX , we conclude that v = |X| |H ∩X| . (3.1) If λ ⩽ 100 or gcd(k, λ) = 1, then by [29, 30], we conclude that D is PG2(3, 2) with parameters (15, 7, 3), and G = A7 with the point-stabiliser H = PSL3(2). Therefore, we can assume that λ > 100 and gcd(k, λ) ̸= 1. Since k(k−1) = λ(v−1) and gcd(k, λ) ̸= 1, we conclude that λ | k, and so by Lemma 2.1(ii), the parameter λ divides |H|. In what follows, assuming that λ > 100 divides k and gcd(k, λ) ̸= 1, we show that there is no flag-transitive and point-primitive automorphism group of a symmetric (v, k, λ) design D with λ prime. Let H0 := H ∩X . Then by [5, Theorem 2 and Proposition 6.1], one of the following holds: (i) H0 is intransitive on Ω = {1, . . . , c}; (ii) H0 is transitive and imprimitive on Ω = {1, . . . , c}; (iii) G = Sc and (c,H) is one of the following: (5,AGL1(5)) , (6,PGL2(5)) , (7,AGL1(7)) , (8,PGL2(7)) , (9,AGL2(3)) , ( 10,A6·22 ) , (12,PGL2(11)) ; (iv) G = A6·2 = PGL2(9) and H is D20 or a Sylow 2-subgroup P of G of order 16; (v) G = A6·2 = M10 and H is AGL1(5) or a Sylow 2-subgroup P of G of order 16; (vi) G = A6·22 = PΓL2(9) and H is AGL1(5)×2 or a Sylow 2-subgroup P of G of order 32; (vii) G = Ac and (c,H) is one of the following: (5,D10), (6,PSL2(5)), (7,PSL2(7)), (8,AGL3(2)), (9, 32·SL2(3)), (9,PΓL2(8)), (10,M10), (11,M11), (12,M12), (13,PSL3(3)), (15,A8), (16,AGL4(2)), (24,M24). S. H. Alavi et al.: Almost simple groups as flag-transitive automorphism groups . . . 557 Since λ is a prime divisor of k, it follows from Lemma 2.1(ii) that λ is a prime divisor of |H|. For the possibilities recorded in (iii) – (vii), we then have λ ∈ {2, 3, 5, 7, 11, 13, 23}, and this violates our assumption that λ > 100, see Table 2. Therefore, H0 is either intran- sitive, or imprimitive. Table 2: The possibilities for λ in cases (iii) – (vii) in Section 3. Line H |H| λ 1 AGL1(5) 2 2 · 5 2, 5 2 PGL2(5) 2 3 · 3 · 5 2, 3, 5 3 AGL1(7) 2 · 3 · 7 2, 3, 7 4 PGL2(7) 2 4 · 3 · 7 2, 3, 7 5 AGL2(3) 2 4 · 33 2, 3 6 A6 · 22 25 · 32 · 5 2, 3, 5 7 PGL2(11) 2 3 · 3 · 5 · 11 2, 3, 5, 11 8 D10 2 · 5 2, 5 9 PSL2(5) 2 2 · 3 · 5 2, 3, 5 10 PSL2(7) 2 3 · 3 · 7 2, 3, 7 11 AGL3(2) 2 6 · 3 · 7 2, 3, 7 12 32 · SL2(3) 23 · 33 2, 3 13 PΓL2(8) 2 3 · 33 · 7 2, 3, 7 14 M10 2 4 · 32 · 5 2, 3, 5 15 M11 2 4 · 32 · 5 · 11 2, 3, 5, 11 16 M12 2 6 · 33 · 5 · 11 2, 3, 5, 11 17 PSL3(3) 2 4 · 33 · 13 2, 3, 13 18 A8 2 6 · 32 · 5 · 7 2, 3, 5, 7 19 AGL4(2) 2 10 · 32 · 5 · 7 2, 3, 5, 7 20 M24 2 10 · 33 · 5 · 7 · 11 · 23 2, 3, 5, 7, 11, 23 21 D20 2 2 · 5 2, 5 22 AGL1(5) 2 2 · 5 2, 5 23 P 24 2 24 AGL1(5)× 2 23 · 5 2, 5 25 P 25 2 Note: In line 23, P is a Sylow 2-subgroup of G = A6 · 2 of order 16. In line 25, P is a Sylow 2-subgroup of G = A6 · 22 of order 32. (I) Suppose that H0 = (Ss×Sc−s)∩Ac is intransitive on Ω = {1, . . . , c} with 1 ⩽ s < c/2. In this case, H = (Ss × Sc−s) ∩ G. Note that H is maximal in G as long as s ̸= c − s. Since λ is an odd prime divisor of |H|, it follows that λ divides s! or (c − s)!, and since s < c− s, we conclude that λ ⩽ max{s, c− s} = c− s. (3.2) Note that H0 contains all the even permutations of H , and hence H0 = H if G = Ac, or the index of H0 in H is 2 if G = Sc. Since G is flag-transitive, H is transitive on the set of blocks passing through α. Hence H fixes exactly one point in P , and so stabilizes exactly 558 Ars Math. Contemp. 23 (2023) #P4.03 / 553–562 one s-subset, say S, in Ω. Therefore, we can identify the point α of P with the unique s- subset S of Ω stabilized by H . Thus v = ( c s ) . Since H0 acting on Ω is intransitive, it has at least two orbits. According to [10, page 82], two points of P are in the same orbit under H0 if and only if the corresponding s-subsets S1 and S2 of Ω intersect S in the same number of points. Thus G acting on P has rank s + 1, and each H0-orbit Oi on P corresponds to a possible size i ∈ {0, 1, . . . , s} and these are precisely the families of s-subsets of Ω that intersect S, see also [1, Proposition 2.5]. Then if di is the length of a G-orbit on P , then d0 = 1, and dj = ( s j−1 )( c−s s−j+1 ) when G = Ac or dj = ( s j−1 )( c−s s−j+1 ) /2 when G = Sc for j = 1, . . . , s. By Lemma 2.1(iii), we have that k divides λdj for all nontrivial subdegrees dj of G. By taking j = s, we have that k divides λs(c− s), and so k ⩽ λs(c− s). As λv < k2 by Lemma 2.1(ii), it follows from (3.2) that v = ( c s ) < λs2(c− s)2 ⩽ s2(c− s)3. Set t := c− s. Thus ( s+ t s ) < s2t3. (3.3) Applying Lemma 2.3(i), we conclude that (3.3) holds only for s ⩽ 8. If s = 1, then v = c ⩾ 5. Note that G is (v− 2)-transitive on P . Since 2 < k ⩽ v− 2, G acts k-transitively on P . Then ( c k ) = |BG| = |B| = v = c for every block B ∈ B. This implies that k = 1 or k = c − 1. Since k ⩾ λ > 100, we conclude that k = c − 1, that is to say, D is a trivial design, which is a contradiction. If s = 2, then the subdegrees are 1, ( c−2 2 ) , 2(c − 2) if G = Ac, or 1, ( c−2 2 ) /2, (c − 2) if G = Sc. Thus G is a primitive permutation group of rank 3. Therefore, the possibilities for D can be read off from [11], which gives no example with λ > 100 prime. If s = 3, then by Lemma 2.1(iii), k divides λd3 = 3λ(c− 3), and so k = 3λ(c− 3)/u for some positive integer u. We apply Lemma 2.1(i) and since v − 1 = ( c 3 ) − 1 = (c − 3)(c2 + 2)/6, we deduce that 3λ(c− 3) u · ( 3λ(c− 3) u − 1 ) = λ(c− 3)(c2 + 2) 6 , and so (c2 + 2)u2 + 18u− 54(c− 3)λ = 0, (3.4) for some positive integer u. Define f(x) := (c2 + 2)x2 + 18x − 54(c − 3)λ with x ⩾ 1. Note here that c > 2s = 6 and λ ⩽ c − 3 by (3.2). Then f ′(x) = 2(c2 + 2)x + 18 > 0 for x ⩾ 1. If x ⩾ 8, then since λ ⩽ c − 3, we have that f(x) = (c2 + 2)x2 + 18x − 54(c − 3)λ ⩾ 10c2 + 324c − 214 > 0 for c > 6. Therefore, we cannot find any positive integer u ⩾ 8 satisfying (3.4), and hence 1 ⩽ u ⩽ 7. Thus by (3.4), we have that 54λ = u2(c + 3) + (11u2 + 18u)/(c − 3), and so c − 3 divides 11u2 + 18u, where 1 ⩽ u ⩽ 7. Moreover, λ > 100 is a prime number less than 665 as λ ⩽ c − 3 ⩽ 11u2 + 18u ⩽ 665. For these values of u, c and λ, it is easy to check that (3.4) does not hold. S. H. Alavi et al.: Almost simple groups as flag-transitive automorphism groups . . . 559 Table 3: An upper bound and a lower bound for t and c when 4 ⩽ s ⩽ 8. s Bounds for t Bounds for c 4 5 ⩽ t ⩽ 373 9 ⩽ c ⩽ 377 5 6 ⩽ t ⩽ 46 11 ⩽ c ⩽ 51 6 7 ⩽ t ⩽ 22 13 ⩽ c ⩽ 28 7 8 ⩽ t ⩽ 14 15 ⩽ c ⩽ 21 8 9 ⩽ t ⩽ 11 17 ⩽ c ⩽ 19 If 4 ⩽ s ⩽ 8, then we apply (3.3) and Lemma 2.3(ii), and so we can find a lower bound and an upper bound for t as in the second column of Table 3. Since also c = t − s, we can find a lower bound and an upper bound for c as in the third column of Table 3. For example, if s = 4, then we take t0 = 374 and observe that ( s+t0 s ) = ( 378 4 ) = 837222750 > 837017984 = 42 ·3743 = s2t30, then Lemma 2.3(ii) implies that if t ⩾ 374, then (3.3) does not hold, which is a contradiction. Thus t ⩽ 373. Moreover, it is easy to check that (3.3) holds for t ⩾ 5. Thus 5 ⩽ t ⩽ 373. Note that t = c − 4, and hence 9 ⩽ c ⩽ 377. This follows the first row of Table 3. For each t, s and c as in Table 3, we note by (3.2) that λ ⩽ c− s = t. Then we obtain v = ( c s ) and all the possibilities for prime λ > 100. But it is easy to check that for such parameters v and λ, we cannot find any possible parameters set (v, k, λ) satisfying Lemma 2.1. (II) Suppose now that H0 is transitive and imprimitive on Ω = {1, . . . , c}. In this case, H = (Ss ≀ Sc/s) ∩ G is imprimitive, where s divides c and 2 ⩽ s ⩽ c/2. Indeed, H0 is transitive and imprimitive on Ω = {1, . . . , c}, H0 acting on Ω preserves a partition Σ of Ω into t classes of size s with t ⩾ 2, s ⩾ 2 and c = st. Thus H0 ⩽ GΣ < G. Since G is isomorphic to Sc or Ac and since both natural actions of G and X on Ω are primitive, we conclude that H0 contains all the even permutations of Ω preserving the partition Σ. By the same argument as in [10, Case 2], [17, (3.2)] and [29, pages 1489-1490], the imprimitive partition Σ is the only nontrivial partition of Ω preserved by H0. Since X acts transitively on all the partitions of Ω into t classes of size s, we can identify the points of the D with the partitions of Ω into t classes of size s, and so v = ( ts s )( (t−1)s s ) · · · ( 3s s )( 2s s ) /(t!), that is to say, v = ( ts− 1 s− 1 )( (t− 1)s− 1 s− 1 ) · · · ( 3s− 1 s− 1 )( 2s− 1 s− 1 ) . (3.5) We note that the suborbits of G on Ω can be described by the notion of j-cyclics intro- duced in [10, page 84]. Indeed, if a partition Σ1 of Ω is a point of P , then for j = 2, . . . , t, the set Γj of j-cyclic partitions with respect to Σ1 is a union of H-orbits on P , see [10, Case 2] and [29, pages 1490-1491]. Therefore, by Lemma 2.1(iii), k divides λds, where ds = { s2 ( t 2 ) , if s ⩾ 3; t(t− 1), if s = 2. (3.6) Therefore, by Lemma 2.1(iii), we have that k divides λds, where ds is as in (3.6). Note that λ is a prime divisor of k, and so by Lemmas 2.1(ii), we conclude that λ ⩽ c = st. 560 Ars Math. Contemp. 23 (2023) #P4.03 / 553–562 If s = 2, then t ⩾ 3 as c = st ⩾ 5. By (3.5), we have that v = ∏t−2 i=0[2t − (2i + 1)] and since k divides λd2 = λt(t− 1) and λ ⩽ c = 2t, it follows from Lemma 2.1(ii) that t−2∏ i=0 [2t− (2i+ 1)] < λd22 ⩽ 2t3(t− 1)2 < 2t5. This forces t ⩽ 6, and hence λ ⩽ 2t = 12, which is a contradiction as λ > 100. If s ⩾ 3, then since,( is− 1 s− 1 ) = is− 1 s− 1 · is− 2 s− 2 · · · is− (s− 1) 1 > is−1 with 2 ⩽ i ⩽ t, by (3.5), we conclude that v > t(s−1)(t−1). Since also k divides λds = λs2 ( t 2 ) and λ ⩽ st, we deduce by Lemma 2.1(ii) that t(s−1)(t−1) < s5t ( t 2 )2 . Thus t(s−1)(t−1)−5 < s5. Note that s ⩾ 3. Then this inequality holds only for t = 2 and 3 ⩽ s ⩽ 30; t = 3 and 3 ⩽ s ⩽ 8; t = 4 and s = 3, 4; t = 5 and s = 3. However, for such pairs (t, s), we easily observe that λ ⩽ st < 100, which is a contradic- tion. This completes the proof. 4 Proof of Corollary 1.2 Let D be a nontrivial symmetric (v, k, λ) design with λ prime. Suppose that G is a flag- transitive and point-primitive automorphism group of D of almost simple type with socle X . Since λ is prime, by [23], the socle X cannot be a sporadic simple group. If the socle X is a simple group of Lie type, then by [3, Theorem 1], D is the point-hyperplane design of PG(n − 1, q) with λ = (qn−2 − 1)/(q − 1) prime and X = PSLn(q), or (D, G) is as in one of the lines 1-3 and lines 5-6 of Table 1. If the socle X is an alternating group Ac of degree c ⩾ 5, then Theorem 1.1 implies that D is the unique design PG2(3, 2) with parameters (15, 7, 3), and G = A7 with the point-stabiliser PSL3(2), and this follows line 4 of Table 1. This finishes the proof. ORCID iDs Seyed Hassan Alavi https://orcid.org/0000-0002-6442-2507 Ashraf Daneshkhah https://orcid.org/0000-0001-8355-6853 References [1] S. H. Alavi, A generalisation of Johnson graphs with an application to triple factorisations, Dis- crete Math. 338 (2015), 2026–2036, doi:10.1016/j.disc.2015.05.001, https://doi.org/ 10.1016/j.disc.2015.05.001. S. H. Alavi et al.: Almost simple groups as flag-transitive automorphism groups . . . 561 [2] S. H. Alavi, M. Bayat and A. Daneshkhah, Symmetric designs admitting flag-transitive and point-primitive automorphism groups associated to two dimensional projective special groups, Des. Codes Cryptography 79 (2016), 337–351, doi:10.1007/s10623-015-0055-9, https:// doi.org/10.1007/s10623-015-0055-9. [3] S. H. Alavi, M. Bayat and A. Daneshkhah, Almost simple groups of Lie type and symmetric designs with λ prime, Electron. J. Comb. 28 (2021), research paper p2.13, 38, doi:10.37236/ 9366, https://doi.org/10.37236/9366. [4] S. H. Alavi, M. Bayat and A. Daneshkhah, Finite exceptional groups of Lie type and sym- metric designs, Discrete Math. 345 (2022), 22, doi:10.1016/j.disc.2022.112894, id/No 112894, https://doi.org/10.1016/j.disc.2022.112894. [5] S. H. Alavi and T. C. Burness, Large subgroups of simple groups., J. Algebra 421 (2015), 187– 233, doi:10.1016/j.jalgebra.2014.08.026, https://doi.org/10.1016/j.jalgebra. 2014.08.026. [6] T. Beth, D. Jungnickel and H. Lenz, Design Theory. Vol. I., volume 69 of Encycl. Math. Appl., Cambridge University Press, Cambridge, 2nd edition, 1999, doi:10.1017/cbo9780511549533, https://doi.org/10.1017/cbo9780511549533. [7] S. Braić, A. Golemac, J. Mandić and T. Vučičić, Primitive symmetric designs with up to 2500 points, J. Comb. Des. 19 (2011), 463–474, doi:10.1002/jcd.20291, https://doi.org/10. 1002/jcd.20291. [8] C. J. Colbourn and J. H. Dinitz (eds.), The CRC handbook of combinatorial designs, Discrete Math. Appl. (Boca Raton), Chapman & Hall/CRC, Boca Raton, FL, 2nd edition, 2007. [9] J. H. Conway, R. T. Curtis, S. P. Norton, R. A. Parker and R. A. Wilson, Atlas of Finite Groups, in: Maximal subgroups and ordinary characters for simple groups. With comput. assist. from J. G. Thackray, Oxford University Press, Eynsha, 1985. [10] A. Delandtsheer, Finite flag-transitive linear spaces with alternating socle, in: Algebraic combi- natorics and applications. Proceedings of the Euroconference, ALCOMA, Gößweinstein, Ger- many, September 12–19, 1999, Springer, Berlin, pp. 79–88, 2001. [11] U. Dempwolff, Primitive rank 3 groups on symmetric designs, Des. Codes Cryptography 22 (2001), 191–207, doi:10.1023/a:1008373207617, https://doi.org/10.1023/a: 1008373207617. [12] J. D. Dixon and B. Mortimer, Permutation Groups, in: Graduate Texts in Mathematics, Springer-Verlag, New York, volume 163, 1996, doi:10.1007/978-1-4612-0731-3, https: //doi.org/10.1007/978-1-4612-0731-3. [13] H. Dong and S. Zhou, Affine groups and flag-transitive triplanes, Sci. China, Math. 55 (2012), 2557–2578, doi:10.1007/s11425-012-4476-x, https://doi.org/10.1007/ s11425-012-4476-x. [14] W. M. Kantor, Primitive permutation groups of odd degree, and an application to finite pro- jective planes, J. Algebra 106 (1987), 15–45, doi:10.1016/0021-8693(87)90019-6, https: //doi.org/10.1016/0021-8693(87)90019-6. [15] P. B. Kleidman, The subgroup structure of some finite simple groups, Ph.D. thesis, University of Cambridge, 1987. [16] E. S. Lander, Symmetric Designs: an Algebraic Approach, volume 74 of London Math- ematical Society Lecture Note Series, Cambridge University Press, Cambridge, 1983, doi: 10.1017/cbo9780511662164, https://doi.org/10.1017/cbo9780511662164. [17] E. O’Reilly Regueiro, Biplanes with flag-transitive automorphism groups of almost simple type, with alternating or sporadic socle., Eur. J. Comb. 26 (2005), 577–584, doi:10.1016/j.ejc. 2004.05.003, https://doi.org/10.1016/j.ejc.2004.05.003. 562 Ars Math. Contemp. 23 (2023) #P4.03 / 553–562 [18] E. O’Reilly Regueiro, On primitivity and reduction for flag-transitive symmetric designs, J. Comb. Theory, Ser. A 109 (2005), 135–148, doi:10.1016/j.jcta.2004.08.002, https://doi. org/10.1016/j.jcta.2004.08.002. [19] E. O’Reilly-Regueiro, Biplanes with flag-transitive automorphism groups of almost sim- ple type, with classical socle, J. Algebr. Comb. 26 (2007), 529–552, doi:10.1007/ s10801-007-0070-7, https://doi.org/10.1007/s10801-007-0070-7. [20] E. O’Reilly-Regueiro, Classification of flag-transitive symmetric designs, in: 6th Czech-Slovak international symposium on combinatorics, graph theory, algorithms and applications, DIMA- TIA Center, Charles University, Prague, Czech Republic, July 10–16, 2006, Elsevier, Am- sterdam, pp. 535–542, 2007, doi:10.1016/j.endm.2007.01.074, https://doi.org/10. 1016/j.endm.2007.01.074. [21] E. O’Reilly-Regueiro, Biplanes with flag-transitive automorphism groups of almost sim- ple type, with exceptional socle of Lie type., J. Algebr. Comb. 27 (2008), 479–491, doi: 10.1007/s10801-007-0098-8, https://doi.org/10.1007/s10801-007-0098-8. [22] C. E. Praeger, The flag-transitive symmetric designs with 45 points, blocks of size 12, and 3 blocks on every point pair, Des. Codes Cryptography 44 (2007), 115–132, doi:10.1007/ s10623-007-9071-8, https://doi.org/10.1007/s10623-007-9071-8. [23] D. Tian and S. Zhou, Flag-transitive 2-(v, k, λ) symmetric designs with sporadic socle, J. Comb. Des. 23 (2015), 140–150, doi:10.1002/jcd.21385, https://doi.org/10.1002/ jcd.21385. [24] Y. Zhang, Z. Zhang and S. Zhou, Reduction for primitive flag-transitive symmetric 2-(v, k, λ) designs with λ prime, Discrete Math. 343 (2020), 4, doi:10.1016/j.disc.2020.111843, id/No 111843, https://doi.org/10.1016/j.disc.2020.111843. [25] S. Zhou and H. Dong, Sporadic groups and flag-transitive triplanes, Sci. China, Ser. A 52 (2009), 394–400, doi:10.1007/s11425-009-0011-0, https://doi.org/10.1007/ s11425-009-0011-0. [26] S. Zhou and H. Dong, Alternating groups and flag-transitive triplanes, Des. Codes Cryp- tography 57 (2010), 117–126, doi:10.1007/s10623-009-9355-2, https://doi.org/10. 1007/s10623-009-9355-2. [27] S. Zhou and H. Dong, Exceptional groups of Lie type and flag-transitive triplanes, Sci. China, Math. 53 (2010), 447–456, doi:10.1007/s11425-009-0051-5, https://doi.org/ 10.1007/s11425-009-0051-5. [28] S. Zhou, H. Dong and W. Fang, Finite classical groups and flag-transitive triplanes, Discrete Math. 309 (2009), 5183–5195, doi:10.1016/j.disc.2009.04.005, https://doi.org/10. 1016/j.disc.2009.04.005. [29] Y. Zhu, H. Guan and S. Zhou, Flag-transitive 2 − (v, k, λ) symmetric designs with (k, λ) = 1 and alternating socle, Front. Math. China 10 (2015), 1483–1496, doi:10.1007/ s11464-015-0480-0, https://doi.org/10.1007/s11464-015-0480-0. [30] Y. Zhu, D. Tian and S. Zhou, Flag-transitive point-primitive (v, k, λ)-symmetric designs with λ at most 100 and alternating socle, Math. Slovaca 66 (2016), 1037–1046, doi:10.1515/ ms-2016-0201, https://doi.org/10.1515/ms-2016-0201. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.04 / 563–575 https://doi.org/10.26493/1855-3974.2905.c94 (Also available at http://amc-journal.eu) A classification of connected cubic vertex-transitive bi-Cayley graphs over semidihedral group* Jianji Cao † School of Applied Mathematics, Shanxi University of Finance and Economics, Taiyuan Shanxi 030006, P. R. China Young Soo Kwon ‡ Department of Mathematics, Yeungnam University, Gyeongsan 38541, R. Korea Mimi Zhang § School of Mathematical Science, Hebei Normal University, Shijiazhuang 050024, P. R. China Received 14 June 2022, accepted 15 January 2023, published online 10 March 2023 Abstract A graph Γ is said to be a bi-Cayley graph over a group H if there exists a subgroup of Aut(Γ) isomorphic to H acting semiregularly on its vertex set with two orbits. In this paper, we give a complete classification of connected cubic vertex-transitive bi-Cayley graphs over semidihedral group. As a byproduct, we construct a family of vertex-transitive non-Cayley graphs. Keywords: Semidihedral group, bi-Cayley graph, vertex-transitive. Math. Subj. Class. (2020): 05C25, 20B25. *The authors would like to thank the referee for his/her valuable suggestions and useful comments contributed to the final version of this paper. †The first author was supported by the China Scholarship Council Foundation (CSC No:201908140049), the National Natural Science Foundation of China (12171302) and the Natural Science Foundation of Shanxi Province (202103021224287). ‡Corresponding author. The second author was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2018R1D1A1B05048450) and (2021K2A9A2A11101586). §The third author was supported by the National Natural Science Foundation of China (12101181) and the Natural Science Foundation of Hebei Province (A2019205180). E-mail addresses: 13994371056@163.com (Jianji Cao), ysookwon@ynu.ac.kr (Young Soo Kwon), 14118412@bjtu.edu.cn (Mimi Zhang) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 564 Ars Math. Contemp. 23 (2023) #P4.04 / 563–575 1 Introduction Throughout this paper, all groups are assumed to be finite, and all graphs are assumed to be finite, connected, simple and undirected. For a graph Γ, let V (Γ), E(Γ) and A(Γ) denote vertex set, edge set and arc set of Γ, respectively. A graph Γ is said to be vertex-transitive, edge-transitive and arc-transitive if the full automorphism group Aut (Γ) acts transitively on V (Γ), E(Γ) and A(Γ), respectively. For other terminology related to group theory and graph theory not defined here, we refer the reader to [1, 11]. Let G be a group and S be a subset of G such that S−1 = S and 1 /∈ S. Then the Cayley graph Γ = Cay(G,S) over G with respect to S is defined as the graph with vertex set V (Γ) = G and edge set E(Γ) = {{g, sg} | g ∈ G, s ∈ S}. Similarly, for a given group H , letR, L and S be subsets ofH such thatR−1 = R, L−1 = L andR∪L does not contain the identity element of H . The bi-Cayley graph over H denoted by BiCay(H , R, L, S) is the graph having vertex set the union of the right part H0 = {h0 | h ∈ H} and the left part H1 = {h1 | h ∈ H}, and edge set the union of the right edges {{h0, g0} | gh−1 ∈ R}, the left edges {{h1, g1} | gh−1 ∈ L} and the spokes {{h0, g1} | gh−1 ∈ S}. When |R| = |L| = s, BiCay(H , R, L, S) is said to be an s-type bi-Cayley graph. The triple (R,L, S) of three subsets R,L, S of a group H is called bi-Cayley triple if R = R−1, L = L−1 and 1 ∈ S. Two bi-Cayley triples (R,L, S) and (R′, L′, S′) of a group H are said to be equivalent, denoted by (R,L, S) ≡ (R′, L′, S′), if either (R′, L′, S′) = (R,L, S)α or (R′, L′, S′) = (L,R, S−1)α for some automorphism α of H . By Proposition 2.1(3)–(4), the bi-Cayley graphs corresponding to two equivalent bi-Cayley triples of the same group are isomorphic. In the study of bi-Cayley graphs, a considerable attention was given to the follow- ing problem: for a given finite group H , classify bi-Cayley graphs over H with specific symmetry properties. For example, vertex-transitive (edge-transitive) generalized Petersen graphs had been classified in [4, 9]. Marušič and Pisanski in [6] classified all cubic arc- transitive bi-Cayley graphs over dihedral group. All tetravalent edge-transitive bicirculants (bi-Cayley group over cyclic group) were characterized in [5], a classification of cubic edge-transitive bi-Cayley graphs over inner abelian p-groups were presented in [10], and all cubic vertex-transitive bi-Cayley graphs over abelian groups were classified in [13]. Recently, Zhang and Zhou in [12] gave a classification of cubic edge-transitive bi-Cayley graphs over dihedral groups. Motivated by the works listed above, in this paper, we shall investigate cubic bi-Cayley graphs over semidihedral groups. Recall that the semidihedral group of order 4n with n an even is defined as follows: SD4n = ⟨a, b | a2n = b2 = 1, b−1ab = an−1⟩. Note that all cubic bi-Cayley graphs over abelian groups have been classified in [13]. So we assume that n ≥ 4. The Petersen graph is a bi-Cayley graph over a cyclic group of order 5, and the Petersen graph is also a vertex-transitive non-Cayley graph. There are many research focusing on the classification of vertex-transitive non-Cayley graphs, see [3, 4, 7, 8]. By Magma, we found some examples of vertex-transitive non-Cayley graph Γ, where Γ is a cubic vertex- trasntive bi-Cayley graph over SD4n. So another motivation for us to consider cubic vertex-transitive bi-Cayley graphs over SD4n is to construct some kind of vertex-transitive non-Cayley graphs. J. Cao et al.: A classification of connected cubic vertex-transitive bi-Cayley graphs . . . 565 In [2] a classification of cubic edge-transitive bi-Cayley graphs over semidihedral group is given. For the completeness of the results, we list the main theorem in [2] in the follow- ing. ( For the definition of CQ(t, n), we refer the reader to [12]) Theorem 1.1 ([2, Theorem 1]). Let Γ be a cubic connected bi-Cayley graph over semidi- hedral group SD4n. Then Γ is edge-transitive if and only if (R,L, S) is equivalent to one of the following triples. Furthermore, all of the corresponding graphs are arc-transitive. (1) (R,L, S) ≡ ({b}, {ba4}, {1, a}) with n = 4 and Γ is isomorphic to F032A. (2) (R,L, S) ≡ ({b}, {ba2}, {1, a}) with n = 6 and Γ is isomorphic to F048A. (3) (R,L, S) ≡ ({b}, {ba2t}, {1, a}), with t an odd, 3 ≤ t ≤ n − 3, n | 2(t2 + t + 1) and Γ is isomorphic to CQ(t, n). (4) (R,L, S) ≡ ({b, ba2}, {a, a−1}, {1}) with n = 4 and Γ is isomorphic to F032A. (5) (R,L, S) ≡ ({b, ba6}, {a, a−1}, {1}) with n = 10 and Γ is isomorphic to F080A. (6) (R,L, S) ≡ ({b, ba2}, {a, a−1}, {1}) with n = 12 and Γ is isomorphic to F096A. In this paper, we determine all cubic vertex-transitive bi-Cayley graphs over semidihe- dral group SD4n. The main results are in the following. Theorem 1.2. Let Γ be a 0-type cubic connected bi-Cayley graph over SD4n. Then Γ is a Cayley graph and Γ is isomorphic to BiCay(SD4n, ∅, ∅, S), where S = {1, a, b}, {1, ba, b} or {1, ba, a}. Theorem 1.3. Let Γ be a 1-type cubic connected bi-Cayley graph over SD4n. Then Γ is a vertex-transitive graph if and only if one of the followings holds. Furthermore all of the corresponding graphs are Cayley graphs. (1) (R,L, S) ≡ ({b}, {bai}, {1, aj}) with i an even, j an odd and the greatest common divisor of i, j and n is equal to 1. (2) (R,L, S) ≡ ({b}, {bal}, {1, ba}) with (l − 1)2 ≡ 1, n− 1 (mod 2n). Theorem 1.4. Let Γ be a 2-type cubic connected bi-Cayley graph over SD4n. Then Γ is a vertex-transitive graph if and only if one of the followings holds: (1) (R,L, S) ≡ ({b, ban}, {ba, ban+1}, {1}); (2) (R,L, S) ≡ ({a, a−1}, {b, ba2l}, {1}) with 2l2 ≡ 2 (mod 2n); (3) (R,L, S) ≡ ({a, a−1}, {b, ba2l}, {1}) with 2l2 ≡ −2 (mod 2n). Furthermore, the graphs corresponding to (1) and (2) are Cayley graphs and the graph corresponding to (3) is a non-Cayley graph. Theorem 1.1 gives a classification of cubic edge-transitive and arc-transitive bi-Cayley graphs over SD4n. Theorems 1.2, 1.3 and 1.4 give a classification of cubic vertex-transitive bi-Cayley graphs over SD4n. As a byproduct, we construct a family of vertex-transitive non-Cayley graphs which correspond to (3) in Theorem 1.4. 566 Ars Math. Contemp. 23 (2023) #P4.04 / 563–575 2 Preliminary In this section, we give two properties of bi-Cayley graph. Proposition 2.1 ([14, Lemma 3.1]). For a bi-Cayley graph BiCay(H,R,L, S) over H , the following hold. (1) H is generated by R ∪ L ∪ S. (2) Up to graph isomorphism, S can be chosen to contain the identity of H . (3) For any automorphism α of H , BiCay(H, R, L, S) ∼= BiCay(H, Rα, Lα, Sα). (4) BiCay(H, R, L, S) ∼= BiCay(H, L, R, S−1). Let Γ = BiCay(H, R, L, S). For an automorphism α of H and x, y, g ∈ H , define two permutations of V (Γ) = H0 ∪H1 as follows: δα,x,y : h0 7→ (xhα)1, h1 7→ (yhα)0, ∀h ∈ H, σα,g : h0 7→ (hα)0, h1 7→ (ghα)1, ∀h ∈ H. Set I = {δα,x,y | α ∈ Aut (H) s.t. Rα = x−1Lx, Lα = y−1Ry, Sα = y−1S−1x}, F = {σα,g | α ∈ Aut (H) s.t. Rα = R, Lα = g−1Lg, Sα = g−1S}. Proposition 2.2 ([14, Theorem 1.1]). Let Γ = BiCay(H, R, L, S) be a bi-Cayley graph over the group H . Then NAut (Γ)(R(H)) = R(H)⋊ F if I = ∅ and NAut (Γ)(R(H)) = R(H)⟨F, δα,x,y⟩ if I ̸= ∅ and δα,x,y ∈ I . Furthermore, for any δα,x,y ∈ I , we have the following: (1) ⟨R(H), δα,x,y⟩ acts transitively on V (Γ); (2) if α has order 2 and x = y = 1, then Γ is isomorphic to the Cayley graph Cay(H̄, R ∪ αS), where H̄ = H ⋊ ⟨α⟩. 3 Proof of main theorems In the beginning of this section, firstly we give some basic properties of SD4n in the fol- lowing lemma without proof which are needed in the proof of our main Theorems. Lemma 3.1. The following hold. (1) SD4n = ⟨a⟩ ∪ b⟨a⟩. Where b⟨a⟩ = {ba2i} ∪ {ba2i+1} with 0 ⩽ i ⩽ n − 1, and furthermore every element of set {ba2i} has order 2 and every element of set {ba2i+1} has order 4. (2) Aut (SD4n) is transitive on sets {ba2i} and {ba2i+1} with 0 ⩽ i ⩽ n− 1. (3) If SD4n = ⟨x, y⟩, then there exists α ∈ Aut (SD4n) mapping {x, y} to one of the following subsets:{a, b}, {ba, b} {ba, a}. J. Cao et al.: A classification of connected cubic vertex-transitive bi-Cayley graphs . . . 567 For any integers i, j satisfying (i, 2n) = 1 and j is even, we have ⟨ai, baj⟩ = ⟨a, b⟩ = SD4n and the map ψi,j : a 7→ ai, b 7→ baj can induce an automorphism of SD4n. In the following of this section we shall use ψi,j to denote the automorphism of SD4n induced by the above map. Proof of Theorem 1.2. Since Γ is a 0-type bi-Cayley graph, R = L = ∅. By Propo- sition 2.1(1) and (2), let S = {1, g, h} with SD4n = ⟨g, h⟩. By Lemma 3.1(3), S is equivalent to one of the following three subsets: {1, a, b}, {1, ba, b}, {1, ba, a}. It is easy to check that ψ−1,0, ψn+1,0 and ψ−1,n+2 are three automorphisms of SD4n of order 2 such that {1, a, b}ψ−1,0 = {1, a, b}−1, {1, ba, b}ψn+1,0 = {1, ba, b}−1 and {1, ba, a}ψ−1,n+2 = {1, ba, a}−1. By Proposition 2.2, Γ is a Cayley graph. □ In order to get the classification of 1-type graph Γ, we give the following Lemma. Lemma 3.2 ([12, Proposition 5.1]). Let K = ⟨a, b | an = b2 = (ab)2 = 1⟩ be a dihedral group of order 2n, and Γ1 = BiCay(K, {b}, {bai}, {1, baj}) be a cubic bi-Cayley graph over K. If Γ1 is vertex-transitive, then (j, n) = 1 and j2 ≡ ±(j − i)2 (mod n). Proof of Theorem 1.3. Assume that Γ is a 1-type cubic vertex-transitive bi-Cayley graph over SD4n. By the definition of 1-type bi-Cayley graph, we can let R = {x}, L = {y} and S = {1, z}. As R = R−1 and L = L−1, both x and y are involutions. Since Γ is connected, by Proposition 2.1(1), SD4n = ⟨x, y, z⟩. We confirm that there is at least one of x and y is not contained in ⟨a⟩. If not, x = y = an implies that SD4n = ⟨an, z⟩ for some z ∈ SD4n, a contradiction. Without loss of generality, assume that x ∈ b⟨a2⟩. By Lemma 3.1(2), Aut (SD4n) acts transitively on the set {ba2k | 0 ≤ k ≤ n − 1}. So we assume that x = b. Now y = an or y = bai for some even i. Case 1: y = an ∈ ⟨a⟩. In this case, z = am or bam. Since SD4n = ⟨x, y, z⟩, one has (m, 2n) = 1. The map am 7→ a, b 7→ b can induce an automorphism α of SD4n, such that (R,L, S)α = ({b}, {an}, {1, a}) or ({b}, {an}, {1, ba}). If (R,L, S) ≡ ({b}, {an}, {1, a}), then (10, 11, (an)1, (an)0, (an+1)1, a1) is the unique 6-cycle passing through 10. On the other hand, there exists a 6-cycle (11, (an)1, (an−1)0, (an−1)1, (a −1)1, (a −1)0) passing through 11 but not passing through 10, contrary to the vertex-transitivity of Γ. Suppose that (R,L, S) ≡ ({b}, {an}, {1, ba}). Then there is a 5-cycle (11, (an)1, (an)0, (ba n+1)1, (ba n+1)0) passing through 11 but not passing through 10. On the other hand, (10, 11, (an)1, (ba)0, (ba)1) and (10, 11, (ban+1)0, (ban+1)1, (ba)1) are all 5-cycles passing through 10, and these also passing through 11, contrary to the vertex-transitivity of Γ. Case 2: y = bai ∈ b⟨a2⟩ for some even i. In this case, z = aj or baj for some odd j. Subcase 2.1: z = aj ∈ ⟨a⟩ for some odd j. In this case, it is easy to check that ψ−1,i is an automorphism of SD4n of order 2 such that Rψ−1,i = L, Lψ−1,i = R and Sψ−1,i = S−1. By Proposition 2.2(2), Γ is a Cayley graph. If (j, 2n) ̸= 1, then ⟨a⟩ = ⟨ai, aj⟩. So the greatest common divisor of i, j and n is equal to 1. This is the graph corresponding to (1) in the theorem. 568 Ars Math. Contemp. 23 (2023) #P4.04 / 563–575 Subcase 2.2: z = baj ∈ b⟨a⟩ for some odd j. Suppose that j = n2 with odd n 2 . By the connectivity of Γ, ⟨a⟩ = ⟨a n 2 , ai⟩, and hence ⟨ai⟩ = ⟨a2⟩ or ⟨a4⟩. By Proposition 2.1(3), we may assume that ai = a2 or a4. Now Γ is isomorphic to a bi-Cayley graph with (R,L, S) ≡ ({b}, {ba2}, {1, ban2 }) or (R,L, S) ≡ ({b}, {ba4}, {1, ban2 }). In the above two cases, we can find that there are two 6-cycles pass- ing through 10, namely (10, b0, b1, (a 3n 2 )0, (ba 3n 2 )0, 11) and (10, b0, (a n 2 )1, (a n 2 )0, (ba n 2 )0, (ba n 2 )1). On the other hand, (11, 10, b0, b1, (a 3n 2 )0, (ba 3n 2 )0) is the unique 6-cycle passing through 11, contrary to the vertex-transitivity of Γ. So, we may assume that z ̸= ba n 2 . Suppose that (j, 2n) ̸= 1. Since ⟨a⟩ = ⟨ai, aj⟩ with i an even and j an odd, the greatest common divisor of i, j and n is equal to 1. We consider the subgraphs of Γ induced by the vertices at distance from 10 and 11 at most 4 respectively. Since j ̸≡ n2 ( mod 2n), one can see that (10, 11, (ban+j)0, (ban+j)1, (an)0, (an)1, (baj)0, (baj)1) is the unique 8-cycle passing through 10 and it is also the unique 8-cycle passing through 11 too. Since (an)0 and (an)1 are the unique vertices that have the longest distance from 10 and 11 in the 8-cycle, respectively, {10, (an)0} and {11, (an)1} are blocks of Aut (Γ) on V (Γ). Let C0 = {{10, (an)0}R(h) | h ∈ SD4n}, C1 = {{11, (an)1}R(h) | h ∈ SD4n} and C = C0 ∪ C1. Then C is a complete block system of Aut (Γ). Let ΓC be the quotient graph. Let N = ⟨R(an)⟩ and let K be the kernel of Aut (Γ) acting on C. Now one can show that K = N and ΓC has valence 3 and K = N is semiregular. Since R(SD4n) acts on V (Γ) semiregularly with two orbits, R(SD4n)/N acts on C semiregularly with two orbits C0 and C1. So the quotient graph ΓC is a bi- Cayley graph over R(SD4n)/N . Let SD4n = R(SD4n)/N and let h̄ = hN for any h ∈ R(SD4n). Now SD4n = ⟨ā, b̄ | ān = b̄2 = (āb̄)2 = 1̄⟩ ∼= D2n. Also we can assume that V (ΓC) = {h̄0 | h̄ ∈ SD4n} ∪ {h̄1 | h̄ ∈ SD4n}. Note that for any h̄ ∈ SD4n NΓC (h̄0) = {bh0, h̄1, bajh1}, NΓC (h̄1) = {baih1, h̄0, ban+jh0}. So we may view ΓC as the bi-Cayley graph BiCay(SD4n, {b̄}, {bai}, {1̄, baj}). Since Γ is vertex-transive, the quotient graph ΓC is also vertex-transitive. Since ΓC is a 1-type cubic vertex-transitive bidihedrant, ΓC is a Cayley graph by [12, Proposition 5.1]. Also by Lemma 3.2, (j, n) = 1 and j2 ≡ ±(j−i)2( mod n). This implies that (j, 2n) = 1, which is contrary to (j, 2n) ̸= 1. So we can assume that (j, 2n) = 1. The map aj 7→ a, b 7→ b can induce an automor- phism β of SD4n, such that (R,L, S)β ≡ ({b}, {bal}, {1, ba}). We consider the subgraphs of Γ induced by the vertices at distance from 10 and 11 at most 4, respectively. Considering possible 8-cycles containing 10 and 11, we see that if l ̸≡ 2, n, n + 2, n2 + 1 (mod 2n), then (10, 11, (ban+1)0, (ban+1)1, (an)0, (an)1, (ba)0, (ba)1) is the unique 8-cycle passing through 10 and it is the unique 8-cycle passing through 11 too. In this case, any automor- phism of Γ sends spokes to spokes. Furthermore, {10, (an)0} and {11, (an)1} are blocks of Aut (Γ) on V (Γ) and by a similar way, one can see that (l − 1)2 ≡ ±1 (mod n). So (l− 1)2 ≡ ±1 (mod 2n), (l− 1)2 ≡ n+ 1 (mod 2n) or (l− 1)2 ≡ n− 1 (mod 2n). It is easy to find that l ≡ 2, n, n+2 (mod 2n) also satisfy (l− 1)2 ≡ ±1 (mod 2n). In the following, we divide the proof into four subcases. J. Cao et al.: A classification of connected cubic vertex-transitive bi-Cayley graphs . . . 569 Subsubcase 2.2.1: (R,L, S) ≡ ({b}, {ban2 +1}, {1, ba}) with n2 an odd. In this case, we can find that there are two 6-cycles passing through 11, namely (11, (ba n 2 +1)1, (ba n 2 +1)0, (a 3n 2 )1, (ba)1, 10) and (11, (ba n 2 +1)1, (a n 2 )0, (a n 2 )1, (ba n+1)1, (ban+1)0). On the other hand, (11, (ba n 2 +1)1, (ba n 2 +1)0, (a 3n 2 )1, (ba)1, 10) is the unique 6-cycle passing through 10, contrary to the vertex-transitivity of Γ. Subsubcase 2.2.2: (R,L, S) ≡ ({b}, {bal}, {1, ba}) with (l − 1)2 ≡ ±1 (mod 2n). If (l−1)2 ≡ −1 (mod 2n), then l2−2l+2 ≡ 0 (mod 2n) implies that n|( l 2 2 − l+1). Since l is even, l 2 2 − l + 1 is odd. This implies that n is also odd, a contradiction. Assume that (l − 1)2 ≡ 1 (mod 2n). Then ((l − 1)2, 2n) = 1, and furthermore (n+1− l, 2n) = 1. It is easy to check that ψn+1−l,l is an automorphism of SD4n of order 2 such that Rψn+1−l,l = L, Lψn+1−l,l = R and Sψn+1−l,l = S−1. By Proposition 2.2(2), Γ is a Cayley graph. This is the graph of type (2) in the theorem. Subsubcase 2.2.3: (R,L, S) ≡ ({b}, {bal}, {1, ba}) with (l − 1)2 ≡ n+ 1( mod 2n). Suppose that Γ is vertex-transitive. Then there is an automorphism ω2 of Γ such that 10 ω2 = 11. Note that 11ω2 = 10 or 11ω2 = (ban+1)0. Suppose that 11ω2 = 10. Then b0ω2 = (bal)1 and (ba l 1) ω2 = b0. We consider the subgraphs of Γ induced by the vertices at distance from 10 and 11 at most 5, respec- tively. It is easy to find that both (bal)0 and (al−1)0 are adjacent with (bal)1, furthermore {b0, (an−1)1}ω2 = {(bal)1, (bal)0} or {b0, (an−1)1}ω2 = {(bal)1, (al−1)0}. Assume that {b0, (an−1)1}ω2 = {(bal)1, (bal)0}. Then (an−1)1 ω2 = (bal)0 and (an−1)0 ω2 = (an+l−1)1. There is a unique 10-cycle passing through vertexes 11, 10, b0, (an−1)1 and (an−1)0, that is (11, 10, b0, (an−1)1, (an−1)0, (ban)1, (ban)0, (an)0, (ban+1)1, (ba n+1)0). On the other hand, there are two 10-cycles passing through vertexes 10, 11, (ba l)1, (ba l)0 and (an+l−1)1, that are (10, 11, (bal)1, (bal)0, (an+l−1)1, (an+l−1)0, (ban+l−1)0, (ba n+l−1)1, (a n−1)1, b0) and (10, 11, (bal)1, (bal)0, (an+l−1)1, (an+l−1)0, (ban+l)1, (a n)1, (ba)0, (ba)1), a contradiction. So {b0, (an−1)1}ω2 = {(bal)1, (al−1)0}, which implies (an−1)1 ω2 = (al−1)0. By a similar reason, one can show that {(bal)1, (al−1)0}ω2 = {b0, (an−1)1}, and hence (al−1)0 ω2 = (an−1)1. Therefore 10 ω2 11 ω2 10, b0 ω2 (bal)1 ω2 b0, (a l−1)0 ω2 (an−1)1 ω2 (al−1)0; Now let us consider the subgraphs of Γ induced by the vertices at distance from (al−1)0 and (an−1)1 at most 5, respectively. We can get (an−1)0 ω2 (al−1)1 ω2 (an−1)0, (ban−1)0 ω2 (ba2l−1)1 ω2 (ban−1)0, (a2(l−1))0 ω2 (a2(n−1))1 ω2 (a2(l−1))0; By a similar way, we can get (ak(n−1))1 ω2 = (ak(l−1))0 for any k. By inserting k = l − 1, we have (a(l−1)(n−1))1 ω2 = (an+1−l)1 ω2 = (an+1)0. On the other hand, (10, 11, (ban+1)0, (ba n+1)1, (a n)0, (a n)1, (ba)0, (ba)1) is the unique 8-cycle passing through 10 and 11 implies that (10, 11, (ban+1)0, (ban+1)1, (an)0, (an)1, (ba)0, (ba)1) is mapped to (11, 10, (ba)1, (ba)0, (a n)1, (a n)0, (ba n+1)1, (ba n+1)0) by ω2. So (ban+1)1 ω2 = (ba)0, furthermore (an+1−l)1 ω2 = a0, a contradiction. 570 Ars Math. Contemp. 23 (2023) #P4.04 / 563–575 Suppose that 11ω2 = (ban+1)0. Then b0ω2 = (bal)1 and (ba l 1) ω2 = (an+1)0. We consider the subgraphs of Γ induced by the vertices at distance from 10 and 11 at most 5, respectively. Since both (an+1)1 and (ban+2)1 are adjacent with (an+1)0, we have {bal1, (al−1)0}ω2 = {(an+1)0, (an+1)1} or {(bal)1, (al−1)0}ω2 = {(an+1)0, (ban+2)1}. Assume that {(bal)1, (al−1)0}ω2 = {(an+1)0, (ban+2)1}. Then (al−1)0 ω2 = (ban+2)1 and (al−1)1 ω2 = (ban+2)0. There is a unique 10-cycle passing through ver- texes 10, 11, (bal)1, (al−1)0 and (al−1)1, that is (10, 11, (bal)1, (al−1)0, (al−1)1, (ban+l)0, (ban+l)1, (a n)1, (ba)0, (ba)1). On the other hand, there are two 10-cycles passing through vertexes 11, (ban+1)0, (an+1)0, (ban+2)1 and (ban+2)0, that are (11, (ban+1)0, (an+1)0, (ban+2)1, (ba n+2)0, a1, a0, (ba)0, (ba)1, 10) and (11, (ban+1)0, (an+1)0, (ban+2)1, (ban+2)0, a1, (ba l+1)1, (a l)0, (ba l)0, (ba l)1), a contradiction. So {(bal)1, (al−1)0}ω2 = {(an+1)0, (an+1)1}, and hence (al−1)0 ω2 = (an+1)1. Therefore ω2 maps the path P1 : 10, 11, (ba l)1, (a l−1)0 to the path Q1 : 11, (ban+1)0, (an+1)0, (an+1)1 in this order, namely 10ω2 = 11, 11ω2 = (ban+1)0, (bal)1 ω2 = (an+1)0 and (al−1)0 ω2 = (an+1)1. Similarly, we consider the subgraphs of Γ induced by the vertices at distance from (al−1)0 and (an+1)1 at most 5, respectively. One can show that ω2 maps the path P2 : (al−1)0, (al−1)1, (ba 2l−1)1, (a 2(l−1))0 to the path Q2 : (an+1)1, (ba2)0, (a2)0, (a2)1 in this order. By a similar way, we can get (ak(l−1))0 ω2 = (ak(n+1))1 for any k. By inserting k = l− 1, we have (a(l−1)(l−1))0 ω2 = (an+1)0 ω2 = (a(l−1)(n+1))1 = (a n+l−1)1. On the other hand, (10, 11, (ban+1)0, (ban+1)1, (an)0, (an)1, (ba)0, (ba)1) is the unique 8-cycle pass- ing through 10 and 11 implies that (10, 11, (ban+1)0, (ban+1)1, (an)0, (an)1, (ba)0, (ba)1) is mapped to (11, (ban+1)0, (ban+1)1, (an)0, (an)1, (ba)0, (ba)1, 10) by ω2. So (ban+1)0 ω2 = (ban+1)1, which implies (an+1)0 ω2 = (an+1−l)1, a contradiction. There- fore Γ is not vertex-transitive, a contradiction. Subsubcase 2.2.4: (R,L, S) ≡ ({b}, {bal}, {1, ba}) with (l − 1)2 ≡ n− 1 (mod 2n). Note that n ≡ 2 (mod 4) in this case. One can check that the map ω3 : (a k)0 7→ (ak(1−l))1, (ak)1 7→ (ban+1+k(1−l))0, (bak)0 7→ (bak(1−l)+l)1, (bak)1 7→ (a(k−1)(1−l))0, with 0 ≤ k < 2n is a permutation on V (Γ) with order 8. Furthermore, for any 0 ≤ k < 2n, we have NΓ((a k)0) ω3 = {(ban+1+k(1−l))0, (ak(1−l))0, (bak(1−l)+l)1} = NΓ((ak(1−l))1), NΓ((a k)1) ω3 = {(ak(1−l))1, (ban+1+k(1−l))1, (an+1+k(1−l))0} = NΓ((ban+1+k(1−l))0), NΓ((ba k)0) ω3 = {(bak(1−l)+l)0, (a(k−1)(1−l))0, (ak(1−l))1} = NΓ((bak(1−l)+l)1), NΓ((ba k)1) ω3 = {(a(k−1)(1−l))1, (bak(1−l)+l)1, (ba(k−1)(1−l))0} = NΓ((a(k−1)(1−l))0). So ω3 induces an automorphism of Γ of order 8. Denote H01 = {(ak)0}, H02 = {(bak)0}, and H11 = {(ak)1}, H12 = {(bak)1} with 0 ≤ k < 2n. We have the following: H01 ω3 H11 ω3 H02 ω3 H12 ω3 H01 Note that ⟨R(a)⟩ acts transitively on the sets H01, H02, H11, H12, respectively. So M2 = ⟨R(a), ω3⟩ is a vertex-transitive subgroup of Aut (Γ). By calculation, ω34 = R(an), ω−13 R(a)ω3 = R(a) 1−l. So M2 = ⟨R(a)⟩⟨ω3⟩ and furthermore |M2| = 8n. Therefore M2 acts regularly on V (Γ), and hence Γ is a Cayley graph of type (2) in the theorem. □ J. Cao et al.: A classification of connected cubic vertex-transitive bi-Cayley graphs . . . 571 Proof of Theorem 1.4. Let Γ be a 2-type bi-Cayley graph and let R = {x1, x2}, L = {y1, y2} and S = {1}. Firstly, we assume that all of x1, x2, y1 and y2 belong to b⟨a⟩. By the structure of SD4n, if their orders are the same, then SD4n ̸= ⟨x1, x2, y1, y2⟩. So without loss of gen- erality, we can assume that x1, x2 have order 2 and y1, y2 have order 4. By Lemma 3.1 (2), Aut (SD4n) acts transitively on set {ba2i} with 0 ⩽ i ⩽ n − 1, we can let x1 = b, x2 = ba 2t, y1 = bas and y2 = ban+s. So (R,L, S) ≡ ({b, ba2t}, {bas, ban+s}, {1}) with s an odd. It is easy to find that (11, (bas)1, (an)1, (ban+s)1) is the unique 4-cycle pass- ing through 11. The vertex-transitivity of Γ implies that there is a unique 4-cycle passing through 10. Considering possible 4-cycles containing 10, we have (a2t)0 = (a−2t)0, and hence n = 2t. Noticing that ⟨a⟩ = ⟨as, an⟩, we get (s, 2n) = 1. So the map f : as 7→ a, b 7→ b can induce an automorphism of SD4n such that (R,L, S)f ≡ ({b, ban}, {ba, ban+1}, {1}). Hence, we can assume that (R,L, S) = ({b, ban}, {ba, ban+1}, {1}). Let Σ = Cay(D8n, {d, dc, dc2n}) where D8n = ⟨c, d | c4n = d2 = 1, dcd = c−1⟩. Define a map from V (Γ) to V (Σ) as follows: ϕ : (ar)0 7→ c2r, (ar)1 7→ dc2r+1, (bar)0 7→ dc2r, (bar)1 7→ c2r−1, with 0 ≤ r ≤ 2n− 1. Furthermore, for any r ∈ Z2n, we have NΓ((a r)0) ϕ = {(ar)1, (bar)0, (ban+r)0}ϕ = {dc2r+1, dc2r, dc2n+2r} = NΣ(c2r), NΓ((a r)1) ϕ = {(ar)0, (bar+1)1, (ban+r+1)1}ϕ = {c2r, c2r+1, c2n+2r+1} = NΣ(dc 2r+1), NΓ((ba r)0) ϕ = {(bar)1, (ar)0, (an+r)0}ϕ = {c2r−1, c2r, c2n+2r} = NΣ(dc2r), NΓ((ba r)1) ϕ = {(bar)0, (an+r−1)1, (ar−1)1}ϕ = {dc2r, dc2n+2r−1, dc2r−1} = NΣ(c 2r−1). It follows that ϕ is an isomorphism from Γ to Σ. Then Γ is a Cayley graph over a dihedral group. This is the graph of type (1) in the theorem. In the following, we assume that there is at least one of x1, x2, y1, y2 belongs to ⟨a⟩. Without loss of generality, let x1 ∈ ⟨a⟩. We divide the proof into two cases: Case 1: |⟨x1⟩| = 2. In this case, x1 = an. The condition R = R−1 implies that x2 is also an element of order 2, and hence x2 = ba2i1 for some integer i1. Since there is an automorphism of SD4n sending a and ba2i1 to a and b, respectively, we can assume x2 = b up to equivalence. If y1 ∈ ⟨a⟩ and y2 /∈ ⟨a⟩, then y1 ̸= y2−1. So L = L−1 implies that y1 = an and y2 = ba2i2 . Then SD4n ̸= ⟨b, ba2i2 , an⟩, contrary to the connectivity of Γ. Similarly, if y2 ∈ ⟨a⟩ and y1 /∈ ⟨a⟩, then one can get a similar contradiction. So we consider the following two subcases: Subcase 1.1: Both y1 and y2 belong to b⟨a⟩. In this case, both y1 and y2 have order 4 by the connectivity of Γ and the structure of SD4n. Let y1 = bal1 and y2 = ban+l1 for some odd integer l1. Now it holds that SD4n = ⟨b, an, al1⟩. This implies that (l1, 2n) = 1. So the map d : al1 7→ a, b 7→ b can induce an automorphism of SD4n such that (R,L, S)d ≡ ({an, b}, {ba, ban+1}, {1}). Hence, we can assume that (R,L, S) ≡ ({an, b}, {ba, ban+1}, {1}) up to equivalence. Let 572 Ars Math. Contemp. 23 (2023) #P4.04 / 563–575 us consider the subgraphs of Γ induced by the vertices at distance from 10 and 11 at most 3, respectively. There are two 7-cycles passing through 10 but not passing through 11 that is (10, b0, b1, (a n−1)1, (ba n)1, (ba n)0, (a n)0) and (10, b0, b1, (a−1)1, (ban)1, (ban)0, (an)0). On the other hand, (11, (ba)1, (ba)0, a0, (an+1)0, (ban+1)0, (ban+1)1) is the unique 7- cycle passing through 11 but not passing through 10, contrary to the vertex-transitivity of Γ. Subcase 1.2: Both y1 and y2 belong to ⟨a⟩. In this case, we can let y1 = ai3 , y2 = a−i3 . Now it holds that SD4n = ⟨b, an, ai3⟩. This implies that (i3, 2n) = 1. So the map e : ai3 7→ a, b 7→ b can induce an automor- phism of SD4n such that (R,L, S)e ≡ ({an, b}, {a, a−1}, {1}). Hence we can assume that (R,L, S) ≡ ({an, b}, {a, a−1}, {1}). It is easy to find that there is a unique 4-cycle (10, b0, (ba n)0, (a n)0) passing through 10. The vertex-transitivity of Γ implies that there is a unique 4-cycle passing through 11. Considering possible 4-cycles containing 11, we have (a2)1 = (a −2)1, and hence n = 2, contrary to n ≥ 4. Case 2: |⟨x1⟩| ≠ 2 In this case, we can let x1 = ai, x2 = a−i. By the connectivity of Γ and the structure of SD4n, if y1 ∈ ⟨a⟩, then y2 /∈ ⟨a⟩ and y1 ̸= y2−1. By the condition L = L−1, we can assume that L = {an, b} up to equivalence. In this case, Γ is isomorphic to a graph of subcase 1.2, by Proposition 2.1(4). So we can assume that both y1 and y2 belong to b⟨a⟩. We divide the proof into the following two subcases: Subcase 2.1: The orders of y1 and y2 are 2. By Lemma 3.1(2), Aut (SD4n) acts transitively on the set {ba2i} for 0 ⩽ i ⩽ n − 1, we can let y1 = b, y2 = ba2j . So (R,L, S) ≡ ({ai, a−i}, {b, ba2j}, {1}). The vertex- transitivity of Γ implies that there must exist an automorphism α in Aut(Γ) such that 1α0 = 11. We consider the subgraphs of Γ induced by the vertices at distance from 10 and 11 at most 5, respectively. There are six 10-cycles passing through edge {10, (ai)0}, {10, (a−i)0}, {11, b1} and {11, (ba2j)1} separately. In the same time, there are eight 10- cycles passing through edge {10, 11}. This implies that {10, (ai)0}α ̸= {10, 11}. Hence {10, (ai)0}α = {11, b1} or {11, (ba2j)1} and {10, (a−i)0}α = {11, b1} or {11, (ba2j)1}. So (ai)0 α = b1 or (ba2j)1 and (a−i)0 α = b1 or (ba2j)1. Without loss of generality, let (ai)0 α = b1. We consider the subgraphs of Γ induced by the vertices at distance from (ai)0 and b1 at most 5, respectively. Similarly, we can find that there exists six 10- cycles passing through edge {(ai)0, (a2i)0} and {b1, (a−2j)1} separately. There are eight 10-cycles passing through edge {(ai)0, (ai)1} and {b1, b0} separately. So by the vertex- transitivity of Γ, {(ai)0, (a2i)0}α = {b1, (a−2j)1}, and hence (a2i)0 α = (a−2j)1. In a similar way, one can see that the cycle (10, (ai)0, (a2i)0, (a3i)0, · · · , (a−i)0) is mapped to the cycle (11, b1, (a−2j)1, (ba−2j)1, · · · , (ba2j)1) by α. So the lengthes of the cycles (10, (a i)0, (a 2i)0, (a 3i)0, · · · , (a−i)0) and (11, b1, (a−2j)1, (ba−2j)1, · · · , (ba2j)1) are the same, and hence ⟨ai⟩ = ⟨aj⟩. Furthermore, SD4n = ⟨ai, a2j , b⟩ = ⟨ai, b⟩ implies that (i, 2n) = (j, 2n) = 1. It is easy to find that the map g : ai 7→ a, b 7→ b can induce an automorphism of SD4n such that (R,L, S)g ≡ ({a, a−1}, {b, ba2l}, {1}) with (l, 2n) = 1. So we can assume that (R,L, S) = ({a, a−1}, {b, ba2l}, {1}) with (l, 2n) = 1 up to equivalence. Now the cycle (10, a0, (a2)0, (a3)0, · · · , (a−1)0) is mapped to the cycle (11, b1, (a−2l)1, (ba −2l)1, · · · , (a2l)1, (ba2l)1) by α. So we can get (a2k)0 α = (a−2kl)1, (a 2k+1)0 α = (ba−2kl)1 (I) J. Cao et al.: A classification of connected cubic vertex-transitive bi-Cayley graphs . . . 573 where 0 ≤ k ≤ n − 1. Note that (a2k)0 and (a2k+1)0 are adjacent with (a2k)1 and (a2k+1)1, respectively, and (a−2kl)1 and (ba−2kl)1 are adjacent with (a−2kl)0 and (ba−2kl)0, respectively. So we can get (a2k)1 α = (a−2kl)0, (a 2k+1)1 α = (ba−2kl)0 (II) where 0 ≤ k ≤ n− 1. By (I) and (II), it is easy to get α : (a2k)0 7→ (a−2kl)1 7→ (a2kl 2 )0. (III) Since (a2k)0 is adjacent with (a2k+1)0, we have (a2k+1)0 α2 = (a2kl 2±1)0. Similarly, by {(a2k+1)0, (a2k+2)0}α 2 = {(a2kl2±1)0, (a2kl 2+2l2)0}, it holds that (a2kl 2+2l2)0 = (a2kl 2±2)0 or (a2kl 2+2l2)0 = (a 2kl2)0. If (a2kl 2+2l2)0 = (a 2kl2)0, then 2l2 ≡ 0 (mod 2n), which implies that n|l2, contrary to l is odd. So (a2kl2+2l2)0 = (a2kl 2±2)0, and hence, 2l2 ≡ ±2 (mod 2n). If 2l2 ≡ 2 (mod 2n), then the map β : (a2k)0 7→ (a−2kl)1, (a2k)1 7→ (a−2kl)0, (a2k+1)0 7→ (ba−2kl)1, (a2k+1)1 7→ (ba−2kl)0, (ba2k)0 7→ (a−2kl+1)1, (ba2k)1 7→ (a−2kl+1)0, (ba2k+1)0 7→ (ban+1−2kl)1, (ba2k+1)1 7→ (ban+1−2kl)0, with 0 ≤ k < n is a permutation on V (Γ) with order 2. Furthermore, for any 0 ≤ k < n we have NΓ((a 2k)0) β = {(ba−2kl)1, (ba2(1−k)l)1, (a−2kl)0} = NΓ((a−2kl)1), NΓ((a 2k)1) β = {(a−2kl+1)0, (a−2kl−1)0, (a−2kl)1} = NΓ((a−2kl)0), NΓ((a 2k+1)0) β = {((a−2kl)1, (a−2(k+1)l)1, (ba−2kl)0} = NΓ((ba−2kl)1), NΓ((a 2k+1)1) β = {(ban−1−2kl)0, (ban+1−2kl)0, (ba−2kl)1} = NΓ((ba−2kl)0), NΓ((ba 2k)0) β = {(ba−2kl+1)1, (ba2(1−k)l+1)1, (a−2kl+1)0} = NΓ((a−2kl+1)1), NΓ((ba 2k)1) β = {(a−2kl+2)0, (a−2kl)0, (a−2kl+1)1} = NΓ((a−2kl+1)0) NΓ((ba 2k+1)0) β = {(an+1−2(k+1)l)1, (an+1−2kl)1, (ban+1−2kl)0} = NΓ((ban+1−2kl)1), NΓ((ba 2k+1)1) β = {(ba−2kl)0, (ba2−2kl)0, (ban+1−2kl)1} = NΓ((ban+1−2kl)0). So β induces an automorphism of Γ of order 2. Denote H01 = {(a2i)0}, H02 = {(a2i+1)0}, H03 = {(ba2i)0}, H04 = {(ba2i+1)0}, H11 = {(a2i)1}, H12 = {(a2i+1)1}, H13 = {(ba2i)1}, H14 = {(ba2i+1)1}, with 0 ≤ i < n. Let H0 = H01 ⋃ H02 ⋃ H03⋃ H04; H1 = H11 ⋃ H12 ⋃ H13 ⋃ H14; We have the following: H01 β H11 R(b) H13 β H02 R(b) H04 β H14 R(b) H12 β H03 R(b) H01. R(a2) acts transitively on Hij where i = 0, 1; j = 1, 2, 3, 4; So M = ⟨R(a2), R(b), β⟩ is a vertex-transitive subgroup of Γ. 574 Ars Math. Contemp. 23 (2023) #P4.04 / 563–575 By calculation, βR(a2)β = R(a−2l) ∈ ⟨R(a2)⟩, so β normalizes ⟨R(a2)⟩. Noticing that R(b) also normalizes ⟨R(a2)⟩, we see that ⟨R(a2)⟩ ⊴ M . We consider the group M/⟨R(a2)⟩ = ⟨β,R(b)⟩ = ⟨βR(b), R(b)⟩. By calculation (βR(b))4 = R(a−2l−2) ∈ ⟨R(a2)⟩ andR(b)2 = 1 imply that βR(b) 4 = R(b) 2 = 1. It is easy to find thatR(b) −1 βR(b) R(b) = βR(b) −1 . So M ≃ D8, and |M | = 8n. Therefore M acts regularly on V (Γ). So Γ is a Cayley graph. This is the graph of type (2) in the theorem. Let 2l2 ≡ −2 ( mod 2n). Now the map γ : (a2k)0 7→ (a−2kl)1, (a2k)1 7→ (a−2kl)0, (a2k+1)0 7→ (ba−2kl)1, (a2k+1)1 7→ (ba−2kl)0, (ba2k)0 7→ (a−2kl−1)1, (ba2k)1 7→ (a−2kl−1)0, (ba2k+1)0 7→ (ban−1−2kl)1, (ba2k+1)1 7→ (ban−1−2kl)0, with 0 ≤ k < n is a permutation on V (Γ). Furthermore, for any 0 ≤ k < n we have NΓ((a 2k)0) γ = {(ba−2kl)1, (ba2(1−k)l)1, (a−2kl)0} = NΓ((a−2kl)1), NΓ((a 2k)1) γ = {(a−2kl+1)0, (a−2kl−1)0, (a−2kl)1} = NΓ((a−2kl)0), NΓ((a 2k+1)0) γ = {((a−2kl)1, (a−2(k+1)l)1, (ba−2kl)0} = NΓ((ba−2kl)1), NΓ((a 2k+1)1) γ = {(ban−1−2kl)0, (ban+1−2kl)0, (ba−2kl)1} = NΓ((ba−2kl)0), NΓ((ba 2k)0) γ = {(ba−2kl−1)1, (ba2(1−k)l−1)1, (a−2kl−1)0} = NΓ((a−2kl−1)1), NΓ((ba 2k)1) γ = {(a−2kl−2)0, (a−2kl)0, (a−2kl−1)1} = NΓ((a−2kl−1)0) NΓ((ba 2k+1)0) γ = {(an−1−2kl)1, (an−1−2(k+1)l)1, (ban−1−2kl)0} = NΓ((ban−1−2kl)1), NΓ((ba 2k+1)1) γ = {(ba−2kl)0, (ba−2−2kl)0, (ban−1−2kl)1} = NΓ((ban−1−2kl)0). Therefore γ induces an automorphism of Γ mapping 10 to 11. So Γ is a vertex-transitive graph. This is the graph of type (3) in the theorem. Now we aim to show that Γ is a non- Cayley graph. Let H be a vertex-transitive subgroup of Aut (Γ) on V (Γ). Then there exists an automorphism φ ∈ H such that 10φ = 11. It is easy to find that 11φ = 10. So φ2 ∈ H10 . Similar with proof of (III), we can get (a2k)0 φ2 = (a2kl 2 )0 = (a −2k)0. Since (a2k+1)0 is adjacent with (a2k)0 and (a2k+2)0, we have (a2k+1)0 φ2 = (a−2k−1)0, where 0 ≤ k ≤ n − 1. The condition (a2k+1)0 φ2 = (a−2k−1)0 implies that a0φ 2 = a−10 . So φ2 ̸= 1 and |H10 | ≥ 2. This implies that H does not act regularly on V (Γ). Since we choose an arbitrary vertex-transitive subgroup H , Γ is a non-Cayley graph. Subcase 2.2: The orders of y1 and y2 are 4. By Lemma 3.1(2), Aut (SD4n) acts transitively on the set {ba2i+1} for 0 ⩽ i ⩽ n− 1, we can let y1 = ba, y2 = ban+1. The condition SD4n = ⟨ai, ba⟩ implies that (i, 2n) = 1. So we can assume (R,L, S) = ({a, a−1}, {ba, ban+1}, {1}) up to equivalence. It is easy to find that (11, (ba)1, (an)1, (ban+1)1) is the unique 4-cycle passing through 11. The vertex- transitivity of Γ implies that there is a unique 4-cycle through 10. Considering possible 4-cycles containing 10, we have (a2)0 = (a−2)0, and hence n = 2, contrary to n ≥ 4. The proof is now complete. □ ORCID iDs Young Soo Kwon https://orcid.org/0000-0002-1765-0806 J. Cao et al.: A classification of connected cubic vertex-transitive bi-Cayley graphs . . . 575 References [1] J. Bondy and U. Murty, Graph Theory with Applications, Elsevier North Holland, New York, 1976. [2] J. Cao, J. Wang and M. Zhang, Cubic edge-transitive bi-Cayley graphs over semidihedral groups, Chin. J. Eng. Math., submitted. [3] H. Cheng, M. Ghasemi and S. Qiao, Tetravalent vertex-transitive graphs of order twice a prime square, Graphs Comb. 32 (2016), 1763–1771, doi:10.1007/s00373-016-1688-9, https:// doi.org/10.1007/s00373-016-1688-9. [4] R. Frucht, J. E. Graver and M. E. Watkins, The groups of the generalized Petersen graphs, Proc. Camb. Philos. Soc. 70 (1971), 211–218. [5] I. Kovács, B. Kuzman, A. Malnič and S. Wilson, Characterization of edge-transitive 4-valent bicirculants, J. Graph Theory 69 (2012), 441–463, doi:10.1002/jgt.20594, https://doi. org/10.1002/jgt.20594. [6] D. Marušič and T. Pisanski, Symmetries of hexagonal graphs on the torus, Croat. Chemica Acta 73 (2000), 969–981. [7] B. McKay and C. E. Praeger, Vertex-transitive graphs that are not Cayley graphs. I, J. Aus- tral. Math. Soc. Ser. A 56 (1994), 53–63, doi:10.1017/S144678870003473x, https://doi. org/10.1017/S144678870003473x. [8] B. McKay and C. E. Praeger, Vertex-transitive graphs that are not Cayley graphs. II, J. Graph Theory 22 (1996), 321–334, doi:10.1002/(SICI)1097-0118(199608)22:4⟨321::AID-JGT6⟩ 3.0.CO;2-N, https://doi.org/10.1002/(SICI)1097-0118(199608)22: 4<321::AID-JGT6>3.0.CO;2-N. [9] R. Nedela and M. Škoviera, Which generalized Petersen graphs are Cayley graphs?, J. Graph Theory 19 (1995), 1–11, doi:10.1002/jgt.3190190102, https://doi.org/10.1002/ jgt.3190190102. [10] Y.-L. Qin and J.-X. Zhou, Cubic edge-transitive bi-Cayley graphs over inner-abelian p-groups, Commun. Algebra 47 (2019), 1973–1984, doi:10.1080/00927872.2018.1527919, https:// doi.org/10.1080/00927872.2018.1527919. [11] H. Wielandt, Finite Permutation Groups, Academic Press, New York, 1964. [12] M.-M. Zhang and J.-X. Zhou, Trivalent vertex-transitive bi-dihedrants, Discrete Math. 340 (2017), 1757–1772, doi:10.1016/j.disc.2017.03.017, https://doi.org/10.1016/j. disc.2017.03.017. [13] J.-X. Zhou and Y.-Q. Feng, Cubic bi-Cayley graphs over abelian groups, Eur. J. Comb. 36 (2014), 679–693, doi:10.1016/j.ejc.2013.10.005, https://doi.org/10.1016/j.ejc. 2013.10.005. [14] J.-X. Zhou and Y.-Q. Feng, The automorphisms of bi-Cayley graphs, J. Comb. Theory, Ser. B 116 (2016), 504–532, doi:10.1016/j.jctb.2015.10.004, https://doi.org/10.1016/j. jctb.2015.10.004. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.05 / 577–590 https://doi.org/10.26493/1855-3974.2853.b51 (Also available at http://amc-journal.eu) Domination and independence numbers of large 2-crossing-critical graphs* Vesna Iršič † , Maruša Lekše Faculty of Mathematics and Physics, University of Ljubljana, Slovenia and Institute of Mathematics, Physics and Mechanics, Ljubljana, Slovenia Mihael Pačnik, Petra Podlogar, Martin Praček Faculty of Mathematics and Physics, University of Ljubljana, Slovenia Received 23 March 2022, accepted 10 January 2023, published online 20 March 2023 Abstract After 2-crossing-critical graphs were characterized in 2016, their most general subfam- ily, large 3-connected 2-crossing-critical graphs, has attracted separate attention. This paper presents sharp upper and lower bounds for their domination and independence numbers. Keywords: Crossing-critical graphs, domination number, independence number. Math. Subj. Class. (2020): 05C10, 05C62, 05C69 1 Introduction The crossing number cr(G) of a graph G is the smallest number of edge crossings in a drawing of G in the plane. The topic has been widely studied, see for example [7, 8, 17, 18, 20]. A graph G is k-crossing-critical if cr(G) ≥ k, but every proper subgraph H of G has cr(H) < k. Note that subdividing an edge or its inverse operation (suppressing a vertex) *The authors were introduced to the structure of 2-crossing-critical graphs in a workshop at the University of Ljubljana, organized by prof. dr. Drago Bokal. We thank him for several illuminating conversations and ideas. We would also like to thank Sandi Klavžar, Alen Vegi Kalamar, and Simon Brezovnik for co-organizing the workshop. We would also like to thank the anonymous referees for valuable suggestions, especially for the idea of how to simplify the proof of Theorem 3.4. †Corresponding author. V. I. was supported by a postdoctoral fellowship at the Simon Fraser University (Canada) and by the Slovenian Research Agency (research core funding P1-0297 and projects J1-2452, J1-1693, N1-0095, N1-0218). E-mail addresses: vesna.irsic@fmf.uni-lj.si (Vesna Iršič), marusa.lekse@imfm.si (Maruša Lekše), mihapac@gmail.com (Mihael Pačnik), petra.podlogar@hotmail.com (Petra Podlogar), mpracek299@gmail.com (Martin Praček) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 578 Ars Math. Contemp. 23 (2023) #P4.05 / 577–590 do not affect the crossing number of a graph. Thus we can restrict our studies to graphs without degree 2 vertices. Under this restriction, Kuratowski’s Theorem tells us that the only 1-crossing-critical graphs are K5 and K3,3. The classification of 2-crossing-critical graphs has been of interest since the 1980s. Partial results on the topic have been reported in [2, 9, 12, 16, 19], and some related results can be found in [1, 10, 13]. Crossing numbers of graphs with a tile structure have been studied in [14, 15]. Finally, Bokal, Oporewski, Richter, and Salazar [6] provided an almost complete characterization of 2-crossing-critical graphs. In particular, they describe a tile structure of large 3-connected 2-crossing-critical graphs (i.e., all but finitely many 3-connected 2-crossing-critical graphs). Recently, the degree properties of crossing-critical graphs have been studied in [3, 5, 11]. The above-mentioned large 3-connected 2-crossing-critical graphs have since attracted separate attention, see [4, 21, 22]. In [21, 22], the Hamiltonicity of these graphs is dis- cussed, and the number of all Hamiltonian cycles is determined. In [4], several additional properties of large 3-connected 2-crossing-critical graphs have been studied. In particular, the number of vertices and edges can be determined from the signature of a graph, and several results regarding their chromatic number, chromatic index, and tree-width are pre- sented. In the present paper, we extend the studies of large 3-connected 2-crossing-critical graphs to their domination and independence numbers. The rest of the paper is organized as follows. In the next section, necessary definitions and known results are listed. In Section 3, the sharp upper and lower bounds for the domi- nation number of large 3-connected 2-crossing-critical graphs are given, while in Section 4 analogous results are proved for their independence number. 2 Preliminaries Let G be a graph. Its vertex set is denoted by V (G) and its edge set by E(G). The (open) neighborhood of a vertex v ∈ V (G) is N(v) = {u ∈ V (G); uv ∈ E(G)} and the closed neighborhood of v is N [v] = {v}∪N(v). Similarly, for D ⊆ V (G), N [D] = ⋃ v∈D N [v] is the closed neighborhood of D. Note also that [n] = {1, . . . , n} and that the reversed sequence of a sequence a is denoted by a. We now recall the definitions of the domination number and the independence number. Definition 2.1. Let G be a graph. A subset D ⊆ V (G) dominates the set of vertices X ⊆ V (G) if X ⊆ N [D]. If N [D] = V (G), then D is a dominating set of G. The domination number γ(G) of a graph G is the size of a smallest dominating set in G. Definition 2.2. Let G be a graph. A subset X ⊆ V (G) is independent if none of the vertices from X are adjacent. The independence number α(G) of the graph G is the size of a largest independent set. In the rest of the section, we recall the characterization of 2-crossing-critical graphs and provide the necessary definitions which help us describe large 3-connected 2-crossing- critical graphs, i.e., graphs studied in this paper. Note that vertices of degrees 1 and 2 do not affect the crossing number, thus the assumption that the minimum degree is at least 3 is reasonable. Note also that V10 is the graph obtained from C10 by adding the five diagonal edges. We quote the following theorem from [6]. The detailed explanation of the terminology used in this theorem is given afterwards. Theorem 2.3 ([6, Theorem 1.1]). Let G be a 2-crossing-critical graph with a minimum degree of at least 3. Then one of the following holds. V. Iršič et al.: Domination and independence numbers of large 2-crossing-critical graphs 579 (i) G is 3-connected, contains a subdivision of V10, and has a very particular twisted Möbius band tile structure, with each tile isomorphic to one of 42 possibilities. (ii) G is 3-connected, does not have a subdivision of V10, and has at most 3 million vertices. (iii) G is not 3-connected and is one of 49 particular examples. (iv) G is 2- but not 3-connected and is obtained from a 3-connected 2-crossing-critical graph by replacing digons with digonal paths. In the present paper, we study graphs from (i), i.e., 3-connected 2-crossing-critical graphs that contain a subdivision of V10. Since 3-connected 2-crossing-critical graphs that do not contain a subdivision of V10 have at most 3 million vertices, we may call graphs from Theorem 2.3(i) large 3-connected 2-crossing-critical graphs or large 3-con 2-cc graphs for short. This abbreviation is used throughout the paper. Note that it would also be interesting to study other subclasses of graphs, especially graphs from (iv). However, like in [4], we restrict our studies to graphs from (i). To understand the tile structure of large 3-con 2-cc graphs, we need the following defi- nitions that first appeared in [14, 15]. Definition 2.4. 1. A tile is a triplet T = (G,λ, ρ), where G is a graph and λ, ρ are sequences of pairwise distinct vertices of G, where no vertex of G appears in both λ and ρ. The left wall of T is λ, and the right wall of T is ρ. 2. A tile drawing is a drawing D of G in the unit square [0, 1] × [0, 1] for which the intersection of the boundary of the square with D contains precisely the images of the left wall λ and the right wall ρ, and these are drawn in {0}×[0, 1] and {1}×[0, 1], respectively, such that the y-coordinates of the vertices are increasing with respect to their orders in the sequences λ and ρ. 3. The tiles T = (G,λ, ρ) and T ′ = (G′, λ′, ρ′) are compatible if |ρ| = |λ′|. 4. A sequence (T0, . . . , Tm) of tiles is compatible if Ti−1 is compatible with Ti for every i ∈ [m]. 5. The join of compatible tiles (G,λ, ρ) and (G′, λ′, ρ′) is the tile, denoted as (G,λ, ρ)⊗ (G′, λ′, ρ′), whose graph is obtained from G and G′ by identifying the sequence ρ term by term with the sequence λ′. The left wall of the obtained tile is λ and the right wall is ρ′. 6. The join ⊗T of a compatible sequence T = (T0, . . . , Tm) of tiles is defined as T0 ⊗ · · · ⊗ Tm. 7. A tile T is cyclically-compatible if T is compatible with itself. For a cyclically- compatible tile T , the cyclization of T is the graph ◦T obtained by identifying the respective vertices of the left wall with the right wall. Cyclization of a cyclically- compatible sequence of tiles is ◦T = ◦(⊗T ). 8. Let T = (G,λ, ρ) be a tile. The right-inverted tile of T is T ↕ = (G,λ, ρ). The left-inverted tile of T is ↕T = (G,λ, ρ). The inverted tile is ↕T ↕ = (G,λ, ρ). The reversed tile is T↔ = (G, ρ, λ). 580 Ars Math. Contemp. 23 (2023) #P4.05 / 577–590 Note that ⊗T in Definition 2.4, 6. is well-defined since ⊗ is associative. Definition 2.5. The set S of tiles consists of tiles obtained as a combination of one of the two frames shown in Figure 1 and one of the 13 pictures shown in Figure 2 in such a way that a picture is inserted into a frame by identifying the two geometric squares. (This can mean subdividing the frame’s square.) A given picture can be inserted into a frame either with the given orientation or with a 180◦ rotation. Figure 1: Both possible frames. Figure 2: All possible pictures. For later need, the red vertices mark a dominating set of each of them. Note that each picture yields either two or four tiles in S. Altogether the set S contains 42 different tiles. For example, in Figure 3 we see that picture V IA yields four different tiles. We can now define the tile structure of graphs that are of our interest. Their definition first appeared in [6]. Definition 2.6. The set T (S) consists of all graphs of the form ◦((⊗T )↕), where T is a sequence (T0,↕ T ↕ 1 , T2, . . . , ↕ T ↕ 2m−1, T2m), where m ≥ 1 and Ti ∈ S for every i ∈ {0, . . . , 2m}. The obtained vertices of degree 2 are suppressed. Suppressing a vertex of degree 2 is the inverse operation to subdividing an edge, and it does not affect the crossing number of a graph. Note that for the case of calculating the domination and independence numbers of graphs, double edges can be replaced with single ones without changing the invariant. V. Iršič et al.: Domination and independence numbers of large 2-crossing-critical graphs 581 Figure 3: All possible tiles that can be obtained from picture V IA. Theorem 2.7 ([6, Theorems 2.18 and 2.19]). Each graph from T (S) is 3-connected and 2-crossing-critical. Moreover, all but finitely many 3-connected 2-crossing-critical graphs are contained in T (S). Theorem 2.7 gives a nice representation of large 3-con 2-cc graphs, i.e., graphs from Theorem 2.3(i). Graphs from the set T (S) can be described as sequences over the alphabet Σ = {L, d,A,B,D,H, I, V } (see [22]). A signature of a tile T is sig(T ) = Pt IdPb Fr, where Pt ∈ {A,B,D,H, V } describes the top path of the picture, Id ∈ {I, ∅} indicates a possible identifier of the picture, Pb ∈ {A,B,D, V, ∅} describes the bottom path of the picture, and Fr ∈ {L, dL} describes the frame. Here, ∅ labels the empty word. See Figure 1 for possible signatures of frames (Fr), Figure 2 for all possible signatures of pictures (Pt IdPb), and Figure 3 for an additional example of how to describe a tile with its signature. For a graph G ∈ T (S), G = ◦((⊗T )↕) = (T0,↕ T ↕1 , T2, . . . ,↕ T ↕ 2m−1, T2m), a signa- ture is defined as sig(G) = sig(T0) sig(T1) · · · sig(T2m). Additionally, #X denotes the number of occurrences of X in sig(G), where X ∈ Σ. Given a tile T , the join of a sequence of k tiles, starting with T and then alternating between T and T ↕, is denoted by k · T . 3 Domination number In this section, we present an upper bound and a lower bound for the domination number of large 3-con 2-cc graphs, including equality cases for both bounds. 3.1 Upper bound Theorem 3.1. If G is a large 3-con 2-cc graph, then γ(G) ≤ #A+#B +#D +#V + 2 ·#H −#AIV −#V IA. 582 Ars Math. Contemp. 23 (2023) #P4.05 / 577–590 Proof. Each vertex lies on at least one picture. Thus, if D ⊆ V (G) dominates all vertices in each picture, then D is a dominating set of G. Inside each picture we have at least one path (by path we mean the top and the bottom path as in the definition of the signature of a tile). We can see that domination of A, B, D, and V requires at least one vertex, while domination of H requires two vertices. The only exceptions are pictures AIV and V IA, where domination of the picture only requires one vertex and not two, which would be the result of the summation of domination numbers of paths A and V . Figure 2 shows all possible pictures with marked smallest dominating sets. Edges between pictures only add edges between vertices and lower the domination number. This means that the domination number has an upper bound of the sum of domi- nation numbers for individual paths. The upper bound from Theorem 3.1 is sharp, which can be seen in the following two examples. They also show that the number of frames L and dL does not affect the upper bound. Example 3.2. Let G1 = n · V BdL, where n ≥ 3 is an odd number. Figure 4 shows a dominating set of size 2n, meaning γ(G1) ≤ 2n. The formula from Theorem 3.1 shows the same, as #A+#B +#D +#V + 2 ·#H −#AIV −#V IA = 2n. Figure 4: Graph G1 with a marked dominating set of size 2n. Note that to obtain the desired graph, vertices a are identified, vertices b are identified, and after this vertices of degree 2 are suppressed. The same simplification of drawings is used for the rest of the paper. Assume γ(G1) < 2n. The Pigeonhole principle says that there exists at least one picture, which is dominated by at most one vertex. Vertices in the corners of the picture can be dominated by vertices from neighboring pictures. The remaining three inner vertices, which we get from B and V and are painted orange in Figure 5, are yet to be dominated. Since these three vertices cannot be dominated by one vertex, we need at least two vertices to dominate this picture, which leads to a contradiction. Therefore γ(G1) ≥ 2n. Figure 5: Picture V B, where we require that the inner vertices, marked orange, are domi- nated by one vertex. From this, it follows that γ(G1) = 2n. Example 3.3. Let G2 = n · AIV L, where n ≥ 3 is an odd number. We can find a dominating set of size n (see Figure 6), thus γ(G2) ≤ n. This also follows from the formula in Theorem 3.1, as #A+#B +#D +#V + 2 ·#H −#AIV −#V IA = n. V. Iršič et al.: Domination and independence numbers of large 2-crossing-critical graphs 583 Figure 6: Graph G2 with a marked dominating set of size n. We next show that γ(G2) ≥ n. Divide the graph G2 into n disjoint subgraphs, as shown in Figure 7. Each subgraph is induced on the closed neighborhood of the degree 3 vertex and is isomorphic to the paw graph. The position of degree 3 vertices in G2 ensures that the obtained n subgraphs are all pairwise disjoint. We notice that the middle vertex of each subgraph (the vertex of degree 3) can only be dominated by one of the vertices in the same subgraph. Hence we must choose at least one vertex from each one of the n disjoint subgraphs, which means that γ(G2) ≥ n. Figure 7: Graph G2 with disjoint subgraphs marked orange and the middle vertex of each subgraph marked blue. Recall that when vertex a is identified, the obtained vertex of degree 2 is suppressed. It follows that γ(G2) = n. 3.2 Lower bound Theorem 3.4. If G is a large 3-con 2-cc graph, then γ(G) ≥ ⌈ 2 3 ·#L ⌉ . Before proving the result, we list two useful observations. Let G be a large 3-con 2-cc graph. 1. Every vertex of G lies on at least one and at most two tiles. 2. All vertices of a picture P can be dominated by a single vertex v only if the picture is V IA. Moreover, in this case, the vertex v only dominates vertices within picture P . Proof of Theorem 3.4. Let G be a 3-con 2-cc graph. We can assume that G only has L frames, since replacing dL frames with L frames means that we contract some edges, which can only decrease the domination number. Replacing dL frames with L frames doesn’t change the number #L. From Observation 1 we know that every vertex of G lies either on only one tile or on two consecutive tiles. If it lies on only one tile, we say that it belongs to that tile. If it lies on two tiles, we say that it belongs to the tile on the right. Hence every vertex belongs to exactly one tile. 584 Ars Math. Contemp. 23 (2023) #P4.05 / 577–590 Let D be a smallest dominating set of G. First we show that for every trinity of consec- utive tiles, there exist at least two vertices from the set D that belong to one of the tiles in the trinity. Figure 8 shows frames of a trinity of tiles (without the pictures). Vertices that are marked red belong to one of the tiles in the trinity. Vertices that are marked with a green circle can only be dominated from one of the vertices that belong to the trinity of tiles. From Observation 2 it follows that we need at least two elements from the set D to dominate all green vertices. Hence there exist at least two vertices from the set D that belong to the trinity of tiles. Figure 8: A trinity of consecutive tiles. Vertices that are marked with a green circle can only be dominated from one of the vertices that belong to this trinity of tiles. Now we can prove that |D| ≥ 23#L. There are #L trinities of consecutive tiles in the graph G, we denote them 1, 2, . . . ,#L. Let D′ = {(d, k) | d ∈ D, d belongs to one of the tiles in the trinity k}. Since each vertex belongs to exactly three trinities of consecutive tiles, it follows that |D′| = 3 · |D|. For every trinity of tiles k, there exist at least two vertices from D that belong to tiles in the trinity k. Hence there exist at least two elements in D′ with sec- ond component k for every k = 1, 2, . . . ,#L. Therefore |D′| ≥ 2 · #L, hence it holds that 3 · |D| ≥ 2 · #L. Because the domination number of G is an integer, it follows that γ(G) ≥ ⌈ 2 3 ·#L ⌉ . The lower bound from Theorem 3.4 is sharp, which can be seen in the following exam- ple. Example 3.5. Let G3 = n ·DDLDDLAIV L, where n ≥ 1 is an odd number. Figure 9 shows the dominating set of size 23 · 3 · n, meaning γ(G3) ≤ 2n. Our formula from Theorem 3.4 shows the same, as ⌈ 2 3 ·#L ⌉ = ⌈ 2 3 · 3 · n ⌉ = 2n. Figure 9: Graph G3 with a marked dominating set of size 2n. Every trinity of consecutive pictures DDLDDLAIV L requires at least two vertices from the dominating set to dominate all the inner vertices of the trinity, which are marked orange in Figure 10. This means that at least two vertices are needed to dominate this trinity of pictures. Therefore γ(G3) ≥ 2n. From this follows that γ(G3) = 2n. V. Iršič et al.: Domination and independence numbers of large 2-crossing-critical graphs 585 Figure 10: Trinity of consecutive pictures DDLDDLAIV L, where we want the inner vertices, marked orange, to be dominated by one vertex. Note that none of these vertices can be dominated by a vertex outside of this trinity of tiles. 4 Independence number In this section, we present sharp upper and lower bounds for the independence number of large 3-con 2-cc graphs. 4.1 Upper bound Theorem 4.1. If G is a large 3-con 2-cc graph, then α(G) ≤ ⌊ |V(G)| 2 ⌋ . Proof. Since all large 3-con 2-cc graphs are Hamiltonian [22], and the independence num- ber of Hamiltonian graphs is at most 12 |V(G)|, we obtain the desired upper bound. The following example shows that the upper bound from Theorem 4.1 is sharp. Example 4.2. Let G4 = n ·HdL, where n ≥ 3 is an odd number. Then |V(G4)| = 6n. Figure 11 shows that we can choose 3n independent vertices from the graph G4, meaning α(G4) ≥ 3n. Figure 11: Graph G4 with a marked independent set of size 3n. Every vertex of graph G4 lies in exactly one picture. Since we can choose at most three independent vertices in each of the n pictures, α(G4) ≤ 3n, which is also the result of Theorem 4.1. Therefore α(G4) = 3n = ⌊ |V(G4)| 2 ⌋ . 4.2 Lower bound Theorem 4.3. If G is a large 3-con 2-cc graph, then α(G) ≥ min{#L+#d, 2 ·#L− 1}. Proof. For every large 3-con 2-cc graph G we can construct the graph G′ from the same frames used for G, without using the pictures. We notice that if we add pictures into the frames in G′ to get the original graph G, we only add vertices and do not connect any 586 Ars Math. Contemp. 23 (2023) #P4.05 / 577–590 vertices that were previously not connected, thus any picture we add can only increase the independence number, therefore α(G) ≥ α(G′). Note that the graph G′ will have the same number of frames L and dL as the initial graph G. Therefore it suffices to prove the proposed lower bound for the graph G′. We distinguish two cases, the first case is if there are only dL frames and the second if there is at least one L frame. Case 1 If there are only dL frames, we can find 2 · #L − 1 independent vertices, as is shown in Figure 12. Note that in this case min{#L+#d, 2 ·#L−1} = 2 ·#L−1. Figure 12: Graph G′ from Case 1 with a marked independent set of size 2 ·#L− 1. Case 2 If there is at least one L frame, then we can choose the independent set based on the following method. Note that double edges can be ignored when studying the independence number. The graph G′ is then composed of 3- and 4-cycles, which are connected with additional edges (marked orange in Figure 13). These additional edges come from where the top and bottom paths of the pictures were in G. To obtain an independent set of appropriate size, we select one vertex from each 3- cycle and two vertices from each 4-cycle. For every 3-cycle, we select the vertex of degree 3 on its left side. If we have two consecutive 3-cycles, the vertices we chose from them are independent. When selecting vertices in the 4-cycles, we consider all consecutive 4-cycles between two 3-cycles and select vertices for the independent set in these 4-cycles from right to left. The 3-cycle on the right of the consecutive 4-cycles determines how we choose the independent set in the right-most 4-cycle, which in turn uniquely determines how we select two independent vertices in each of these 4-cycles (in the same manner as in Figure 12). Notice that the 3-cycle on the left of these 4-cycles gives no restriction on the selected vertices. Figure 13: An example of the graph G′ from Case 2 with a marked independent set of size #L+#d. The edges that connect the 3- and 4-cycles are marked orange. We have thus chosen two vertices in each 4-cycle and one vertex in each 3-cycle. Since the number of 3-cycles is #L − #d and the number of 4-cycles is #d, we have found an independent set of size (#L−#d) + 2 ·#d = #L+#d. Note that in this case min{#L+#d, 2 ·#L− 1} = #L+#d. V. Iršič et al.: Domination and independence numbers of large 2-crossing-critical graphs 587 The following two examples show that the lower bound from Theorem 4.3 is sharp. The first example naturally follows from the proof of Theorem 4.3, while the second example provides a non-trivial family of sharpness examples. Additionally, examples are selected in such a way that different parts of the minimum are attained. Example 4.4. Let G5 be a large 3-con 2-cc graph built from tiles DDdL and DDL, so that not all of the tiles are DDdL. From Theorem 4.3 we know that α(G5) ≥ #L+#d. Similarly as in the proof, we can find #d 4-cycles and #L−#d 3-cycles in G5, so that every vertex lies on exactly one of them. Every 4-cycle is formed by the two vertices on the right of a DDdL tile and the two vertices on the left of the next tile to the right. Every 3-cycle is formed by the two vertices on the right of a DDL tile and the two vertices on the left of the next tile. Two of those vertices are identified, thus giving us a 3-cycle. The 3-cycles and 4-cycles are marked in Figure 14. Figure 14: Graph G5 with marked 3-cycles and 4-cycles. We can choose at most one independent vertex from every 3-cycle and at most two independent vertices from every 4-cycle, therefore α(G5) ≤ 2 · #d + 1 · (#L − #d) = #L+#d. From this it follows that α(G5) = #L+#d. Example 4.5. Let G6 be a large 3-con 2-cc graph that is built from DDdL, V IAdL, and AIV dL tiles, but not all tiles are V IAdL, and not all tiles are AIV dL. From Theorem 4.3 we know that α(G6) ≥ 2 · #L − 1. We can find at most two independent vertices in each of the tiles DDdL, V IAdL, and AIV dL, therefore we can find at most 2 ·#L independent vertices in G6. For contradiction suppose that α(G6) ̸= 2 · #L − 1, meaning α(G6) = 2 · #L. We try to construct an independent set A with 2 ·#L vertices. Set A must include exactly two vertices from every tile because otherwise set A would have to include at least 3 vertices from one tile, which is impossible. There are two different ways in which we can choose two independent vertices from a DDdL tile, and three different ways for tiles V IAdL and AIV dL. All options are shown in Figure 15. Even though tiles V IAdL and AIV dL have a third option for the choice of two inde- pendent vertices (where the selected vertices are not diagonal), we can’t choose the vertices in set A in this way, since we know that we have to choose two independent vertices from every tile. If we choose the top and bottom right vertex in a V IAdL tile, then the only way to choose two vertices in the next tile is if that tile is also a V IAdL tile and we choose the top and bottom right vertices. We continue this for all tiles, but since not all tiles are V IAdL, at some point we are not able to choose two independent vertices in the next tile. For the same reason, we also cannot choose the two vertices on the left of an AIV dL tile. 588 Ars Math. Contemp. 23 (2023) #P4.05 / 577–590 Figure 15: Tiles DDdL, V IAdL, and AIV dL with two independent vertices marked. This means that for all tiles, the two vertices that are included in set A are the diagonal ones, without loss of generality we can assume that those diagonal vertices in the first tile are the bottom left and the top-right vertex. This choice determines which vertices we must choose in the tile to the right and so on, as is shown in Figure 16. Figure 16: Graph G′6 was constructed from the same frames used for G6, without using the pictures. The first tile of graph G′6 determines which two vertices are included in set A for all other tiles. When we get to the last tile, we get a contradiction (marked orange). When we get to the last tile we get a contradiction. Because of the tile to the left, the only possible vertices from the last tile that can be included in A are the bottom left and the top-right vertex. But the top-right vertex is connected to a vertex in the first tile that is already included in set A, therefore set A cannot include two vertices from the last tile. This means that the independent set A that has 2 · #L elements cannot exist and α(G6) = 2 ·#L− 1. ORCID iDs Vesna Iršič https://orcid.org/0000-0001-5302-5250 References [1] L. Beaudou, C. Hernández-Vélez and G. Salazar, Making a graph crossing-critical by multiply- ing its edges, Electron. J. Comb. 20 (2013), research paper p61, 14, www.combinatorics. org/ojs/index.php/eljc/article/view/v20i1p61. [2] G. S. Bloom, J. W. Kennedy and L. V. Quintas, On crossing numbers and linguistic structures, Graph theory in: Lecture Notes in Math., vol. 1018 (1983). V. Iršič et al.: Domination and independence numbers of large 2-crossing-critical graphs 589 [3] D. Bokal, M. Bračič, M. Derňár and P. Hliněný, On degree properties of crossing-critical fam- ilies of graphs, Electron. J. Comb. 26 (2019), research paper p1.53, 28, doi:10.37236/7753, https://doi.org/10.37236/7753. [4] D. Bokal, M. Chimani, A. Nover, J. Schierbaum, T. Stolzmann, M. H. Wagner and T. Wiedera, Properties of large 2-crossing-critical graphs, J. Graph Algorithms Appl. 26 (2022), 111–147, doi:10.7155/jgaa.00585, https://doi.org/10.7155/jgaa.00585. [5] D. Bokal, Z. Dvořák, P. Hliněný, J. Leaños, B. Mohar and T. Wiedera, Bounded degree con- jecture holds precisely for c-crossing-critical graphs with c ≤ 12, in: 35th International Symposium on Computational Geometry, SoCG 2019, Portland, Oregon, USA, June 18–21, 2019. Proceedings, Wadern: Schloss Dagstuhl – Leibniz-Zentrum für Informatik, p. 15, 2019, doi:10.4230/LIPIcs.SoCG.2019.14, id/No 14, https://doi.org/10.4230/LIPIcs. SoCG.2019.14. [6] D. Bokal, B. Oporowski, R. B. Richter and G. Salazar, Characterizing 2-crossing-critical graphs, Adv. Appl. Math. 74 (2016), 23–208, doi:10.1016/j.aam.2015.10.003, https:// doi.org/10.1016/j.aam.2015.10.003. [7] M. Chimani, P. Kindermann, F. Montecchiani and P. Valtr, Crossing numbers of beyond-planar graphs, Theor. Comput. Sci. 898 (2022), 44–49, doi:10.1016/j.tcs.2021.10.016, https:// doi.org/10.1016/j.tcs.2021.10.016. [8] K. Clancy, M. Haythorpe and A. Newcombe, A survey of graphs with known or bounded crossing numbers, Australas. J. Comb. 78 (2020), 209–296, https://ajc.maths.uq. edu.au/pdf/78/ajc_v78_p209.pdf. [9] G. Ding, B. Oporowski, R. Thomas and D. Vertigan, Large non-planar graphs and an ap- plication to crossing-critical graphs, J. Comb. Theory, Ser. B 101 (2011), 111–121, doi: 10.1016/j.jctb.2010.12.001, https://doi.org/10.1016/j.jctb.2010.12.001. [10] P. Hliněny, Crossing-number critical graphs have bounded path-width, J. Comb. Theory, Ser. B 88 (2003), 347–367, doi:10.1016/S0095-8956(03)00037-6, https://doi.org/10. 1016/S0095-8956(03)00037-6. [11] P. Hliněný and M. Korbela, On the achievable average degrees in 2-crossing-critical graphs, Acta Math. Univ. Comen. 88 (2019), 787–793, http://www.iam.fmph.uniba.sk/ amuc/ojs/index.php/amuc/article/view/1178. [12] M. Kochol, Construction of crossing-critical graphs, Discrete Math. 66 (1987), 311–313, doi:10.1016/0012-365X(87)90108-7, https://doi.org/10.1016/0012-365X(87) 90108-7. [13] J. Leaños and G. Salazar, On the additivity of crossing numbers of graphs, J. Knot Theory Ram- ifications 17 (2008), 1043–1050, doi:10.1142/S0218216508006531, https://doi.org/ 10.1142/S0218216508006531. [14] B. Pinontoan and R. B. Richter, Crossing numbers of sequences of graphs. II: Planar tiles, J. Graph Theory 42 (2003), 332–341, doi:10.1002/jgt.10097, https://doi.org/10. 1002/jgt.10097. [15] B. Pinontoan and R. B. Richter, Crossing numbers of sequences of graphs. I: General tiles, Australas. J. Comb. 30 (2004), 197–206, https://ajc.maths.uq.edu.au/pdf/30/ ajc_v30_p197.pdf. [16] B. Richter, Cubic graphs with crossing number two, J. Graph Theory 12 (1988), 363–374, doi:10.1002/jgt.3190120308, https://doi.org/10.1002/jgt.3190120308. [17] F. Shahrokhi, L. A. Székely and I. Vrt’o, Crossing numbers of graphs, lower bound techniques and algorithms: A survey, in: R. Tamassia and I. G. Tollis (eds.), Graph Drawing, Springer 590 Ars Math. Contemp. 23 (2023) #P4.05 / 577–590 Berlin Heidelberg, Berlin, Heidelberg, 1995 pp. 131–142, doi:10.1007/3-540-58950-3_364, https://doi.org/10.1007/3-540-58950-3_364. [18] A. C. Silva, A. Arroyo, R. B. Richter and O. Lee, Graphs with at most one crossing, Discrete Math. 342 (2019), 3201–3207, doi:10.1016/j.disc.2019.06.031, https://doi.org/10. 1016/j.disc.2019.06.031. [19] J. Širáň, Infinite families of crossing-critical graphs with a given crossing number, Discrete Math. 48 (1984), 129–132, doi:10.1016/0012-365X(84)90140-7, https://doi.org/10. 1016/0012-365X(84)90140-7. [20] L. A. Székely, An optimality criterion for the crossing number, Ars Math. Contemp. 1 (2008), 32–37, doi:10.26493/1855-3974.43.506, https://doi.org/10.26493/1855-3974. 43.506. [21] A. Vegi Kalamar, T. Žerak and D. Bokal, Counting hamiltonian cycles in 2-tiled graphs, Mathematics 9 (2021), doi:10.3390/math9060693, https://doi.org/10.3390/ math9060693. [22] T. Žerak, Hamiltonian Cycles in Large 2-crossing-critical Graphs, Master’s thesis, University of Maribor, 2019. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.06 / 591–603 https://doi.org/10.26493/1855-3974.3061.5bf (Also available at http://amc-journal.eu) Enumerating symmetric pyramids in Motzkin paths* Rigoberto Flórez † Department of Mathematical Sciences, The Citadel, Charleston, SC, U.S.A. José L. Ramı́rez ‡ Departamento de Matemáticas, Universidad Nacional de Colombia, Bogotá, Colombia Received 4 February 2023, accepted 29 March 2023, published online 7 April 2023 Abstract A path in the first quadrant of the xy-plane that starts at the origin having North- East steps (X), Horizontal steps (Z), South-East steps (Y ), and that ends on the x-axis is called Motzkin. A maximal pyramid is a subpath of the form XhZmY h that cannot be extended to Xh+1ZmY h+1. It is symmetric if it cannot be extended to any of these subpaths: Xh+1ZmY h or XhZmY h+1. We use generating functions to enumerate sym- metric pyramids and give the asymptotic behavior of the number of symmetric pyramids. Additionally, we give combinatorial arguments to count some of the mentioned aspects. Keywords: Motzkin path, generating function, symmetric pyramid. Math. Subj. Class. (2020): 05A15, 05A19 1 Introduction A path in the first quadrant of the xy-plane that starts at the origin having North-East steps (X), Horizontal steps (Z), South-East steps (Y ), and that ends on the x-axis is called Motzkin. Whoever is familiar with Dyck paths may verify that Motzkin paths are a gener- alization of them. The set of all Motzkin paths of length n is denoted by Mn and the set ∪∞n=0Mn is denoted by M. A maximal pyramid of weight h in a Motzkin path is a subpath of the *The authors are grateful to an anonymous referee for helpful comments. †The first author was partially supported by the Citadel Foundation, Charleston, SC. ‡The second author was partially supported by Universidad Nacional de Colombia, Project No. 53490. He started working on this project when he was in a short research visit at The Citadel. E-mail addresses: rigo.florez@citadel.edu (Rigoberto Flórez), jlramirezr@unal.edu.co (José L. Ramı́rez) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 592 Ars Math. Contemp. 23 (2023) #P4.06 / 591–603 form XhZmY h that cannot be extended to Xh+1ZmY h+1, with h ≥ 1 and m ≥ 0. It is symmetric if it can not be extended to any of these subpaths: Xh+1ZmY h or XhZmY h+1. There are two types of distinguishable maximal pyramids: if m = 0, the pyramid is called a triangular pyramid, denoted by h or by if there is no ambiguity, see the left-hand side of Figure 1; and if m > 0, the pyramid is called truncated, denoted by h or by if there is no ambiguity, see the right-hand side of Figure 1. The symmetric weight of a path is the sum of the weights of its symmetric pyramids. The height of a pyramid is the y-coordinate of its highest point. That is, the height is measured from the x-axis to the highest point. The symmetric height of a path is the sum of the heights of its symmetric pyramids. A hump of a Motzkin path is a subpath of the form XZkY for any non-negative integer k. If k = 0, it is called sharp hump (or sharp peak). Notice that every pyramid has a hump. Motivated in part by the work done by Asakly [1], Flórez and Ramı́rez [8] introduced the concept of symmetric and asymmetric peaks in Dyck paths, see also Elizalde et al. [5], Flórez et al. [7], and Sun et al. [14]. Following up these works we generalize the symmetric concept to Motzkin paths. Using generating functions we enumerate symmetric pyramids and give the asymptotic behavior of the number of symmetric pyramids. Additionally, we give combinatorial arguments to count some of the mentioned aspects. Our results here, in particular, apply to recover the known results for Dyck paths. Figure 1: Triangular symmetric pyramid and truncated symmetric pyramid. 2 Motzkin paths, humps, and valleys If ck = 1k+1 ( 2k k ) is the kth Catalan number, then the number of Motzkin paths of length n, |Mn|, is given by the nth Motzkin number defined as mn := ⌊n/2⌋∑ k=0 1 k + 1 ( n 2k )( 2k k ) = ⌊n/2⌋∑ k=0 ck ( n 2k ) . (2.1) See, for example, the sequence A001006 of [12]. In this section we give a bivariate generating function enumerating Motzkin paths with respect to the length and the number of humps. The results here are, in fact, lemmas for the main results studied in the upcoming sections. For example, in Section 3 we compare the number of symmetric pyramids with respect to the number of humps. In particular, we prove that these two quantities are in a proportion of 1/2 (see Corollary 3.8). We denote by humps(M) the number of humps in a Motzkin path M . We now in- troduce a bivariate generating function to count the number of humps with respect to the length of a Motzin path: H(q, x) := ∑ M∈M qhumps(M)x|M |. R. Flórez et al.: Enumerating symmetric pyramids in Motzkin paths 593 Proposition 2.1. The generating function for Motzkin paths with respect to the number of humps is given by H(q, x) = 1− 2x+ 2x2 − qx2 − √ (1− 4x+ (4− q)x2)(1− qx2) 2x2(1− x) . Proof. A Motzkin path M can be decomposed as —using first return decomposition— λ (the empty path), ZM , or XM1YM2, where M1 and M2 are Motzkin paths, possibly empty. Notice that for the last decomposition, humps(M) = humps(M1)+humps(M2) —unless M1 is the path Zk, for any k ≥ 0, in which case humps(M) = 1+humps(M2). Therefore, the generating function H(q, x) satisfies this functional equation H(q, x) = 1 + xH(q, x) + x2 ( H(q, x)− 1 1− x + q 1− x ) H(q, x). Solving this functional equation for H(q, x) we obtain the desired result. Let us denote by hmn the total number of humps in Mn. Then we have ∂ ∂q H(q, x) |q=1= ∑ n≥0 hmnx n = 1− 2x+ x2 − (1− x) √ 1− 2x− 3x2 2(1− x)2 √ 1− 2x− 3x2 . The sequence hmn is given by the combinatorial sum (cf. [13, 3]) hmn = 1 2 ∑ j≥0 ( n j )( n− j j ) − 1 2 . Notice that if ( n,3 n ) denotes the central trinomial coefficient, that is, the nth coefficient of the expansion of (x2 + x+ 1)n, then hmn = 12 ( n,3 n ) − 1. The central trinomial coefficient has the asymptotic approximation (cf. [11])( n, 3 n ) ∼ 3 n 2 √ 3 πn . Therefore, the sequence hmn has this asymptotic approximation hmn ∼ 3n 4 √ 3 πn . (2.2) A weak valley of a Motzkin path is any subpath of the form ZZ, Y X,ZX , or Y Z, see Figure 2. Let wv(M) be the number of weak valleys in a Motzkin path M . Let us denote by e(M) the number of horizontal steps in M . We now introduce a multivariate generating function to count the number of weak valleys and horizontal steps, with respect to the length of a Motzin path: Mwv(q, t, x) := ∑ M∈M qwv(M)te(M)x|M |. Figure 2: Weak valleys in a Motzkin path. 594 Ars Math. Contemp. 23 (2023) #P4.06 / 591–603 We need this proposition for our main theorem of Section 3. See [2] for more results about valleys of Motzkin paths. Proposition 2.2. The generating function for Motzkin paths with respect to the number of weak valleys and horizontal steps is given by: Mwv(q, t, x) = 1− tqx− x2 + qx2 − √ (1− tqx− x2 + qx2)2 − 4qx2(1 + tx− tqx) 2qx2 . Proof. From the first return decomposition, a Motzkin path M can be decomposed as λ, ZM1 or XM2YM3, where Mi ∈ M, for i = 1, 2, 3. Notice that humps(M) = humps(M1)+1 and humps(M) = humps(M2)+humps(M3)+1, unless M1 and M3 are the empty paths, in this case, we have that humps(M) = 0 and humps(M) = humps(M1), respectively. Therefore, the generating function Mwv(q, t, x) satisfies the functional equation Mwv(q, t, x) = 1 + tx(qMwv(q, t, x)− q + 1) + x2Mwv(q, t, x)(qMwv(q, t, x)− q + 1). Solving this functional equation for Mwv(q, t, x) we obtain the desired result. The series expansion of the generating function Mwv(q, 1, x) is 1 + x+ (1 + q)x2 + (1 + 2q + q2)x3 + (1 + 4q + 3q2 + q3)x4 + (1 + 6q + 9q2 + 4q3 + q4)x5 + (1 + 9q + 19q2 + 16q3 + 5q4 + q5)x6 +O(x7). The array coefficients of the above series correspond to the sequence A110470. 3 Symmetric pyramids of Motzkin paths In this section we give a multivariate generating function to count the number of symmetric pyramids, the number of triangular, and truncated symmetric pyramids with respect to the length of a Motzkin path. We give an asymptotic behavior of symmetric pyramids. In the end of the section we analyze the proportion existing between the total number of symmetric pyramids and the total number of humps in Mn, when n → ∞. Let us denote by sp(M), stp(M), and strp(M) the number of symmetric pyramids, the number of symmetric triangular pyramids, and the number symmetric truncated pyra- mids, respectively, of a Motzkin path M . It is clear that sp(M) = stp(M) + strp(M). We now introduce a multivariate generating function with respect to the defined parameters: Msp(q, p, t, x) := ∑ M∈M qstp(M)pstrp(M)te(M)x|M |. Theorem 3.1 gives an expression for this multivariate generating function. The result of this theorem is based on marking or distinguishing elements within the generating function. For more details about the method used in the theorem see, for example, these books: Goulden and Jackson [9, page 128] or Flajolet and Sedgewick [6, page 209]. R. Flórez et al.: Enumerating symmetric pyramids in Motzkin paths 595 Theorem 3.1. The generating function for Motzkin paths with respect to the number of symmetric triangular pyramids, the number symmetric truncated pyramids, and the number of horizontal steps is given by: Msp(q, p, t, x) = (1− x2)(1− tx) ( p1 − √ p2 ) 2x2(1− tx− qx2 + (1− p+ q)tx3)2 , where p1 and p2 are these multivariate polynomials p1 = 1− 2tx− (q − t2)x2 + (2− p+ q)tx3 + (1− q − t2)x4 − (p− q)tx5 and p2 = (1−x2)(1−(1+2t)x−(1+q−t2)x2−(1−q−2t+pt−qt−t2)x3+(p−q)tx4) × (1 + (1− 2t)x− (1 + q − t2)x2 + (1− q + 2t− pt+ qt− t2)x3 − (p− q)tx4). Proof. Consider the set G of all Motzkin paths in which an arbitrary number of symmetric pyramids (some, possibly none, possibly all) have been marked. Any path in G can be obtained from a Motzkin path M by inserting pyramids (triangular or truncated) with a marked hump. These pyramids can be inserted in the node of a weak valley, in the initial or final node of M . These kinds of nodes are called insertion nodes (see [4]). Let ins(M) be the number of insertion nodes of M . It is clear that ins(M) = wv(M) + 2 —unless M is the empty path, in which case ins(M) = 1. Consider the generating function Mins(q, t, x) = ∑ M∈M qins(M)te(M)x|x|. Then, it is clear that Mins(q, t, x) = q 2(Mwv(q, t, x)− 1) + q. Let M be a marked Motzkin path in G. We denote by mark1(M) and mark2(M) the num- ber of marked triangular pyramids and truncated pyramids of M , respectively. Consider the generating function G(q, p, t, x) = ∑ M∈G qmark1(M)pmark2(M)te(M)x|M |. Since the elements of G are obtained from elements of M by inserting marked pyramids in the insertions nodes, it translates in terms of generating functions as G(q, p, t, x) = Mins ( 1 1− qx2/(1− x2)− ptx3/((1− x2)(1− tx)) , t, x ) . Finally, the Motzkin paths with marked symmetric pyramids are generated by these substi- tutions: q 7→ q + 1 and p 7→ p+ 1 within Msp(q, p, t, x). That is, we obtain the equality G(q, p, t, x) = Msp(q + 1, p+ 1, t, x) or equivalently Msp(q, p, t, x) = G(q − 1, p− 1, t, x). Using the above equality and Proposition 2.2 we obtain the desired result. 596 Ars Math. Contemp. 23 (2023) #P4.06 / 591–603 The series expansion of the generating function Msp(q, p, t, x) is 1 + tx+ (q + t2)x2 + (pt+ 2qt+ t3)x3 + (q + q2 + 3pt2 + 3qt2 + t4)x4 + (2t+ pt+ 2qt+ 2pqt+ 3q2t+ 6pt3 + 4qt3 + t5)x5 +O(x6). Figure 3 shows some Motzkin paths. Their corresponding weights are shown in the previ- ous expansion in boldface. qx4 pt2x4 pt2x4 q2x4 qt2x4 pt2x4 qt2x4 qt2x4 t4x4 Figure 3: Motzkin paths and the symmetric pyramid statistic. We use sn to denote the total number of symmetric pyramids in Mn. Corollary 3.2. The generating function for Motzkin paths with respect to the number of symmetric pyramids is given by: (1− x)2(1 + x) ( 1− 2x+ (1− q)x2 + 2x3 − qx4 − (1− x2) √ p(x, q) ) 2x2(1− x− qx2 + x3)2 , where p(x, q) = 1 − 4x + 2(2 − q)x2 + 4qx3 − (4 − q2)x4. Moreover, the sequence sn has the generating function ∂ ∂q Msp(q, q, 1, x) |q=1= 1− 4x3 + 4x+ x4 + (1− x2 − 2x)(1− x) √ 1− 2x− 3x2 2(1− x)3(1 + x) √ 1− 2x− 3x2 . These are the first 11 values of the sequence sn for n = 1, 2, . . . , 11: 0, 1, 3, 9, 23, 60, 156, 415, 1121, 3076, 8540. We use s∗k, g ∗ k, and t ∗ k to denote the number of all first symmetric pyramids, all first symmetric triangular pyramids, and all first truncated symmetric pyramids at a ground level, respectively, in Mk. Note that given a path P ∈ Mk, there is another path Q ∈ Mk that is symmetric to P —the first seen from left-to-right and the second seen from from right-to-left. Therefore, using this symmetry of paths in Mk we have that s∗k, g∗k, and t∗k also count the number of all last symmetric pyramids, all last symmetric triangular pyramids, and all last truncated symmetric pyramids at ground level, respectively. Lemma 3.3. If ck is the kth Catalan number, then s∗k = g∗k + t∗k, where g∗k = 2 k−1∑ i=1 (−1)i+1 ( i⌊ i−1 2 ⌋)(k − 1 i+ 1 ) and t∗k = 2 k−1∑ i=1 ( ⌊ k−i2 ⌋∑ j=0 cj ⌊ i− 1 2 ⌋( k − i 2j )) . R. Flórez et al.: Enumerating symmetric pyramids in Motzkin paths 597 Proof. Let us find g∗k. A Motzkin path µk can be decomposed as iµk−i for i = 0, 1, . . . , k − 1, where 0 = ∅. This implies that the number of that type of pyramid is given by g∗k =  ⌊ k−12 ⌋∑ i=1 m2i, if k is even; ⌊ k−12 ⌋∑ i=1 m2i−1, if k is odd. This formula coincides with the sequence A082397 in OEIS multiplied by 2. Therefore, the expression given in the statement is from A082397. We now find t∗k. A Motzkin path of the form µk can be decomposed as iµk−i, for i = 0, 1, . . . , k− 1, where 0 = ∅. This implies that the number of that type of pyramids, in the set of all paths of the form iµk−i, is given by t∗k = ∑k−1 i=1 ⌊ i−1 2 ⌋ mk−i. This and (2.1) give the result in the statement of this lemma. Clearly, the number of first symmetric pyramids is the sum of these two quantities. From the first return decomposition of a Motzkin path we have this particular expres- sion Zµn−1 and XµkY µn−2−k, with µi ∈ Mi and µ0 = λ. (3.1) Here we say that XµkY is the first primitive subpath. We need this decomposition again for the proof of Theorem 4.4. Theorem 3.4. Let sn be the total number of symmetric pyramids in Mn. Then sn satisfies s1 = 0, s2 = 1, s3 = 3, and for n > 3 sn = sn−1 + sn−2 + sn−3 + n−2∑ k=0 mn−2 + n−2∑ k=2 ( (sk − 2s∗k)mn−k−2 + sn−k−2mk ) . Proof. From the decomposition given in (3.1) the number of symmetric pyramids in a path of the form Zµn−1 is given by the number of symmetric pyramids in µn−1, that is counted by sn−1. We now analyze XµkY µn−1−k, for k = 0, 1, . . . , n − 1. If k = 0 or 1, then 1 or 1 becomes the first symmetric pyramid in the path. These give that the total number of symmetric pyramids in the set of all paths of the form XY µn−2 and XZY µn−3 is given by mn−2 + sn−2 +mn−3 + sn−3. Let 0 < k < n be a fixed number. Then the number of symmetric pyramids in the set of paths of the form XµkY µn−1−k is given by (sk − 2s∗k)mn−2−k + sn−2−kmk. Thus, sn−2−kmk counts all symmetric pyramids after the first primitive subpath and (sk − s∗k)mn−2−k counts all symmetric pyramids in the first primitive subpath. Note that skmn−2−k counts all symmetric pyramids in µk. However, the first and the last pyra- mids at the ground level in µk are counted by sk as symmetric, but in XµkY they are not symmetric. So, we must subtract all of them; and that is what s∗k does (see Lemma 3.3). Therefore, for k = 1, 2, . . . , n − 1 we have that the number of symmetric pyramids in paths of the form XµkY µn−1−k is given by ∑n−1 k=1((sk − 2s∗k)mn−k−2 + sn−k−2mk). Now adding the results from each decomposition and varying k from 0 to n− 1, we obtain the desired result. 598 Ars Math. Contemp. 23 (2023) #P4.06 / 591–603 Remark 3.5. Notice that the generating function of the number of symmetric pyramids in Mn is algebraic, then the counting sequence sn satisfies a recurrence relation with poly- nomial coefficients. This can be automatically solved with Kauers’s algorithm [10]. In particular we obtain that sn satisfies the recurrence relation p0(n)sn + p1(n)sn+1 + p2(n)sn+2 + p3(n)sn+3 + p4(n)sn+4 + p5(n)sn+5 = 0, for n ≥ 5, where p0(n) = n 3 − 3n2 − 3n; p1(n) = −3n3 + 9n2 + 15n− 26; p2(n) = −2n3 + 12n2 − 6n− 34; p3(n) = 6n3 − 24n2 − 18n+ 86; p4(n) = n 3 − 9n2 + 9n+ 34; p5(n) = −3n3 + 15n2 + 3n− 60. Theorem 3.6. An asymptotic approximation for sn is given by: sn ∼ 3n 8 √ 3 πn . Proof. The generating function of the sequence sn (see Corollary 3.2) can be written as A(x) +B(x), where A(x) = ∑ n≥0 anx n = 1− 2x− x2 2(1− x)2(1 + x) and B(x) = ∑ n≥0 bnx n = −1 + 3x+ 3x2 − x3 2(1− x)2(1 + x) √ 1− 2x− 3x2 . The sequence an satisfies a0 = 1 and an = −2⌊(n − 1)/2⌋ + 1, for all n ≥ 1. Then, an ∼ −n/2. On the other hand, the main singularity of B(x) is 1/3. From the singularity analysis described in [6] we obtain that bn ∼ 3 n 8 √ 3 πn . Therefore, sn ∼ bn. Comparing the nth coefficients of the generating functions A(x) and B(x) defined in Theorem 3.6 we obtain this corollary. Corollary 3.7. For all n ≥ 0, we have sn = 1 2 n∑ i=0 ( n, 3 i ) bn−i − ⌊ n− 1 2 ⌋ − 1 2 , where b0 = −1 and bn = (n((n+ 1) mod 2) + 2(n mod 2))(−1)n(n−1)/2, for all n ≥ 1. The trinomial coefficient ( n,3 i ) can be calculated using the established combinatorial sum ( n,3 i ) = ∑n k=0 ( n k )( k i−k ) . From Theorem 3.6 and (2.2) we can also conclude the following asymptotic expression. Corollary 3.8. The asymptotic proportion between the number of symmetric pyramids and the number of humps is lim n→∞ sn hmn = 1 2 . R. Flórez et al.: Enumerating symmetric pyramids in Motzkin paths 599 From Theorem 3.1, setting t = 0, we obtain the distribution of symmetric pyramids (symmetric peaks) over the Dyck lattice paths. A Dyck path is a Motzkin path without horizontal steps. The number of Dyck paths of semi-length n is given by the Catalan number cn = 1n+1 ( 2n n ) . The generating function of the Dyck paths with respect to the number of symmetric peaks and semi-length n is given by Msp(q, 0, 0, x1/2). The explicit expression is (1− x) ( 1 + x2 − qx(1 + x)− √ t(x, q) ) 2x(1− qx)2 , where t(x, q) = (1− x)(1− (3+ 2q)x− (1− 4q− q2)x2 − (1− q)2x3), see [4, Theorem 2.1]. Moreover, the generating function of the total number of symmetric peaks over all Dyck paths of semi-length n is (see [8, Theorem 2.3]) ∂ ∂q Msp(q, 0, 0, x) |q=1= −1 + 5x+ (1− x) √ 1− 4x 2(1− x) √ 1− 4x . 4 Symmetric triangular pyramids In this section we give asymptotic approximations for the proportions between the different aspects that we have been studying in this paper. Thus, we give the proportion between the number of symmetric triangular pyramids and these quantities: the number of pyramids, the number of humps, and the number of sharp peaks. We also discuss the combinatorial behavior of triangular pyramids —symmetric pyramids, symmetric weight, and symmetric height. We use gn to denote the number of symmetric triangular pyramids in Mn. Theorem 4.1. The generating function for Motzkin paths with respect to the number of symmetric triangular pyramids is G(x) := ∂ ∂q Msp(q, 1, 1, x) |q=1 . More precisely, G(x) is given by 1− 6x+ 9x2 + 4x3 − 13x4 + 2x5 + 3x6 − (1− 4x+ x2)(1− x)2(1 + x) √ p(x) 2(1− x)3(1 + x)2(1− 3x) , where p(x) = 1− 2x− 3x2. These are the first 11 values of the sequence gn for n = 1, 2, . . . , 11: 0, 1, 2, 6, 14, 37, 96, 259, 706, 1955, 5464. Theorem 4.2. The sequence gn has this asymptotic approximation gn ∼ 3n 4 √ 3 πn . Let spn be the number of sharp humps in Mn. Brennan and Mavhung [2] proved that spn ∼ 3 n 6 √ 3 πn . 600 Ars Math. Contemp. 23 (2023) #P4.06 / 591–603 Corollary 4.3. If sn, gn, hmn, and spn represent the number of symmetric pyramids, triangular pyramids, humps, and sharp humps of Mn, respectively, then these hold: lim n→∞ gn sn = 2 3 ; lim n→∞ gn hmn = 1 3 ; and lim n→∞ gn spn = 1 2 . We recall that mi is the number of Motzkin paths of length i. This theorem gives a recursive relation for the sequence gn. Theorem 4.4. If g∗k is as in Lemma 3.4, then the sequence gn satisfies g1 = 0, g2 = 1, and for n ≥ 3 gn = gn−1 + gn−2 +mn−2 + n−1∑ k=1 ( (gk − 2g∗k)mn−k−2 + gn−k−2mk ) . Proof. This proof is similar to the proof of Theorem 3.4. Replace sk by gk and instead of s∗k use Lemma 3.4 with g ∗ k. 4.1 Weight and height of the symmetric triangular pyramids We use wn to denote the total weight of symmetric triangular pyramids in Mn. In Proposi- tion 4.5 we give a recursive relation to find the values of wn. Its proof is based on a similar argument as in Theorem 4.4. So, we omit it. Proposition 4.5. The sequence wn satisfies that w1 = 0, w2 = 1, and for n ≥ 3 wn = wn−1 + wn−2 +mn−2 + n−2∑ k=1 (( wk + (k + 1 mod 2)− 2 ⌊ k−12 ⌋∑ i=1 imk−2i ) mn−k−2 +mkwn−k−2 ) . These are the first 11 values of the sequence wn, for n = 1, 2, . . . , 11: 0, 1, 2, 7, 16, 44, 112, 303, 818, 2258, 6282. We use hn to denote the symmetric height of triangular pyramids in Mn. Proposition 4.6. Let gk be number of symmetric triangular pyramids in Mk. The sequence hn satisfies h1 = 0, h2 = 1, h3 = 2, and for n ≥ 4 hn = hn−1 + hn−2 +mn−2 + n−2∑ k=1 (( hk + gk − 2 ⌊ k−12 ⌋∑ i=1 (i+ 1)mk−2i ) mn−k−2 +mkhn−k−2 ) . These are the first 11 values of the sequence hn, for n = 1, 2, . . . , 11: 0, 1, 2, 7, 16, 45, 118, 331, 930, 2673, 7744. R. Flórez et al.: Enumerating symmetric pyramids in Motzkin paths 601 5 Symmetric truncated pyramids In this section we give asymptotic approximations for the proportions between the different aspects that we have been studying in this paper. Thus, we give the proportion between the number of symmetric truncated pyramids and these quantities: the number of pyramids and the number of humps. We also discuss the combinatorial behavior of truncated pyramids —symmetric pyramids, symmetric weight, and symmetric height. We use tn to denote the number of symmetric truncated pyramids in the family of Motzkin paths of length n. Theorem 5.1. The generating function for Motzkin paths with respect to the number of symmetric truncated pyramids is T (x) := ∂ ∂p Msp(1, p, 1, x) |p=1 . In particular, if G(x) denotes the generating function of the symmetric triangular pyramids, see Theorem 4.1, then T (x) = x 1− x G(x). These are the first 11 values of the sequence tn for n = 1, 2, . . . , 11: 0, 0, 1, 3, 9, 23, 60, 156, 415, 1121, 3076. From the equality given in Theorem 5.1 we have the relation tn = ∑n−1 i=0 gi. Theorem 5.2. The sequence tn has this asymptotic approximation tn ∼ 3n 8 √ 3 πn . Corollary 5.3. If sn, tn, and hmn represent the number of symmetric pyramids, symmetric truncated pyramids, and humps of Mn, respectively, then these hold: lim n→∞ tn sn = 1 3 and lim n→∞ tn hmn = 1 6 . In Theorem 5.4 we give a recursive relation to calculate the values of the sequence tn. Its proof uses the same technique used in the proof of Theorem 4.4. Theorem 5.4. If t∗k is as in Lemma 3.4, then the sequence tn satisfies t0 = t1 = t2 = 0, t3 = 1, and for n ≥ 4 tn = 2tn−1 − tn−4 +mn−3 + n−2∑ k=2 ((tk − 2t∗k)(mn−k−2 −mn−k−3) +mk(tn−k−2 − tn−k−3)). We use vn to denote the total weight of symmetric truncated pyramids in Mn. We define Ob(i, j) := ⌊ i−1 2 ⌋ (⌊ i−1 2 ⌋ + j ) . The proofs of the following two results are similar to the proofs of Theorems 4.4 and 5.4. Therefore, we omit them. 602 Ars Math. Contemp. 23 (2023) #P4.06 / 591–603 Proposition 5.5. The sequence vn satisfies v1 = v2 = 0, v3 = 1, and for n ≥ 4 vn = vn−3 + vn−2 + vn−1 + n−3∑ k=0 mk + n−2∑ k=2 (( vk + ⌊ k − 1 2 ⌋ − k−1∑ i=2 Ob(i, 1)mk−i ) mn−k−2 + vn−k−2mk ) . These are the first 11 values of the sequence wn for n = 1, 2, . . . , 11: 0, 1, 3, 10, 26, 70, 182, 485, 1303, 3561, 9843. The symmetric truncated height in Mn is the sum of the heights of all symmetric truncated pyramids in Mn. Proposition 5.6. Let kn be the symmetric truncated height in Mn and let ti be the number of symmetric truncated pyramids in Mi. Then the sequence kn satisfies k1 = k2 = 0, k3 = 1, and for n ≥ 4 kn = kn−1 + kn−2 + kn−3 + n−3∑ j=0 mj + n−2∑ j=2 (( kj + tj − j−1∑ i=2 Ob(i, 3)mj−i ) mn−j−2 +mjkn−j−2 ) . These are the first 11 values of the sequence kn for n = 1, 2, . . . , 11: 0, 0, 1, 3, 10, 26, 71, 189, 520, 1450, 4123. ORCID iDs Rigoberto Flórez https://orcid.org/0000-0002-3644-9358 José L. Ramı́rez https://orcid.org/0000-0002-8028-9312 References [1] W. Asakly, Enumerating symmetric and non-symmetric peaks in words, Online J. Anal. Comb. 13 (2018), 7, id/No 2, web.math.rochester.edu/misc/ojac/vol13/169.pdf. [2] C. Brennan and S. Mavhungu, Peaks and valleys in Motzkin paths, Quaest. Math. 33 (2010), 171–188, doi:10.2989/16073606.2010.491177, https://doi.org/10.2989/ 16073606.2010.491177. [3] Y. Ding and R. R. X. Du, Counting humps in Motzkin paths, Discrete Appl. Math. 160 (2012), 187–191, doi:10.1016/j.dam.2011.08.018, https://doi.org/10.1016/j. dam.2011.08.018. [4] S. Elizalde, Symmetric peaks and symmetric valleys in Dyck paths, Discrete Math. 344 (2021), 13, doi:10.1016/j.disc.2021.112364, id/No 112364, https://doi.org/10.1016/j. disc.2021.112364. R. Flórez et al.: Enumerating symmetric pyramids in Motzkin paths 603 [5] S. Elizalde, R. Flórez and J. L. Ramı́rez, Enumerating symmetric peaks in non-decreasing Dyck paths, Ars Math. Contemp. 21 (2021), 23, doi:10.26493/1855-3974.2478.d1b, id/No 4, https://doi.org/10.26493/1855-3974.2478.d1b. [6] P. Flajolet and R. Sedgewick, Analytic Combinatorics, Cambridge University Press, Cambridge, 2009, doi:10.1017/cbo9780511801655, https://doi.org/10.1017/ cbo9780511801655. [7] R. Flórez, L. Junes and J. L. Ramı́rez, Counting asymmetric weighted pyramids in non- decreasing Dyck paths, Australas. J. Comb. 79 (2021), 123–140, https://ajc.maths. uq.edu.au/pdf/79/ajc_v79_p123.pdf. [8] R. Flórez and J. L. Ramı́rez, Enumerating symmetric and asymmetric peaks in Dyck paths, Dis- crete Math. 343 (2020), 9, doi:10.1016/j.disc.2020.112118, id/No 112118, https://doi. org/10.1016/j.disc.2020.112118. [9] I. P. Goulden and D. M. Jackson, Combinatorial Enumeration, Dover, 2004. [10] M. Kauers, Guessing Handbook. Technical Report 09-07, RISC-Linz, 2009. [11] J. Li, Asymptotic estimate for the multinomial coefficients, J. Integer Seq. 23 (2020), article 20.1.3, 9, https://cs.uwaterloo.ca/journals/JIS/VOL23/Li/li14.html. [12] N. J. A. Sloane, The On-Line Encyclopedia of Integer Sequences, {http://oeis.org/}. [13] A. Regev, Humps for Dyck and for Motzkin paths, 2010, arXiv:1002.4504 [math.CO]. [14] Y. Sun, W. Shi and D. Zhao, Symmetric and asymmetric peaks or valleys in (partial) dyck paths, Enumer. Comb. Appl. 2 (2022), article #S2R24, doi:10.54550/eca2022v2s3r24, https: //doi.org/10.54550/eca2022v2s3r24. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.07 / 605–613 https://doi.org/10.26493/1855-3974.3072.3ec (Also available at http://amc-journal.eu) The core of a vertex-transitive complementary prism Marko Orel * University of Primorska, FAMNIT, Glagoljaška 8, Koper, Slovenia and University of Primorska, IAM, Muzejski trg 2, Koper, Slovenia and IMFM, Jadranska 19, Ljubljana, Slovenia Received 20 February 2023, accepted 15 March 2023, published online 12 May 2023 Abstract The complementary prism ΓΓ̄ is obtained from the union of a graph Γ and its comple- ment Γ̄ where each pair of identical vertices in Γ and Γ̄ is joined by an edge. It generalizes the Petersen graph, which is the complementary prism of the pentagon. The core of a vertex-transitive complementary prism is studied. In particular, it is shown that a vertex- transitive complementary prism ΓΓ̄ is a core, i.e. all its endomorphisms are automorphisms, whenever Γ is a core or its core is a complete graph. Keywords: Graph homomorphism, core, complementary prism, self-complementary graph, vertex- transitive graph. Math. Subj. Class. (2020): 05C60, 05C76 1 Introduction In the study of graph homomorphism a basic object is a core (a.k.a. unretractive graph), which is a graph such that all its endomorphisms are automorphisms. A subgraph Γ′ of a graph Γ is its core, if there exists some graph homomorphism φ : Γ → Γ′ and Γ′ is a core. Equivalently, Γ′ is the minimal retract of Γ (cf. [17]). Despite that each graph has its core, which is unique up to isomorphism, it can be often very difficult to determine if a given graph is a core or not (cf. [6, 16, 30]). From this point of view, graphs that have either high degree of symmetry (i.e. ‘large’ automorphism group) or some ‘nice’ combinatorial prop- erties are the most interesting. Many of such classes of graphs are core-complete, which *The author thanks the anonymous referees for the helpful comments that improved the presentation of the paper. This work is supported in part by the Slovenian Research Agency (research program P1-0285 and research projects N1-0140, N1-0208, N1-0210, J1-4084, and N1-0296). E-mail address: marko.orel@upr.si (Marko Orel) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 606 Ars Math. Contemp. 23 (2023) #P4.07 / 605–613 means that they are either cores or their cores are complete graphs. Among them we can find all non-edge-transitive graphs [6, Corollary 2.2], connected regular graphs with the au- tomorphism group that acts transitively on unordered pairs of vertices at distance two [16, Theorem 4.1], and all primitive strongly regular graphs [36, Corollary 3.6]. Given a core- complete graph, it can be extremely complicated to decide if the graph is a core or its core is complete. For some graphs, this task is equivalent to some of the longstanding open prob- lems in finite geometry (see [6, 30]). A well known core is the Petersen graph, which has both ‘large’ automorphism group and ‘nice’ combinatorial properties. Given a family of graphs that (naturally) generalize the Petersen graph it is interesting to study if its members are cores or not. Kneser graphs K(n, r), with 2r < n, are all cores [15, Theorem 7.9.1]. The graph HGLn(F4), whose vertex set is formed by all n× n invertible hermitian matri- ces over the field with four elements and with the edge set {{A,B} : rank(A − B) = 1}, is a core whenever n ≥ 2 [29]. The core of a generalized Petersen graph G(n, k) was stud- ied very recently in [13]. The complementary prism ΓΓ̄, whose definition in full details is given in Section 2, is another generalization of the Petersen graph, which is obtained if Γ is the 5-cycle C51. Graph ΓΓ̄ was introduced in [18] and is the main mater of research in several papers (see for example [1, 3, 7, 10, 19, 26]). Recall that C5 is strongly reg- ular vertex-transitive self-complementary graph. The core of ΓΓ̄ was recently studied by the author [28] (see also the arXiv version [31]). In particular, it was shown that ΓΓ̄ is a core whenever Γ is strongly regular and self-complementary. In this paper we build on a result from [28] and investigate vertex-transitive self-complementary graphs Γ. These are precisely the graphs that provide vertex-transitive complementary prisms [31, Corol- lary 3.8]. For such graphs we prove that ΓΓ̄ is a core whenever Γ is core-complete (see Theorem 3.3), and state an open problem, which asks if there exists a vertex-transitive self- complementary graph Γ such that ΓΓ̄ is not a core (Problem 3.6). The main results are presented in Section 3. In Section 2 we recall some tools and definitions that we need in what follows. 2 Preliminaries All graphs in this paper are finite and simple. The vertex set and the edge set of a graph Γ are denoted by V (Γ) and E(Γ), respectively. A subset of pairwise adjacent vertices in V (Γ) is a clique, while a set of pairwise nonadjacent vertices in V (Γ) is an independent set. The clique number ω(Γ) and the independence number α(Γ) are the orders of the largest clique and the largest independent set in Γ, respectively. In particular, α(Γ) = ω(Γ̄), where Γ̄ is the complement of the graph Γ. The chromatic number of a graph is denoted by χ(Γ). It is well known that χ(Γ) ≥ ω(Γ) and χ(Γ) ≥ nα(Γ) , where n = |V (Γ)| (cf. [5]). A graph homomorphism between graphs Γ1,Γ2 is a map φ : V (Γ1) → V (Γ2) such that {φ(u), φ(v)} ∈ E(Γ2) whenever {u, v} ∈ E(Γ1). If in addition φ is bijective and {u, v} ∈ E(Γ1) ⇐⇒ {φ(u), φ(v)} ∈ E(Γ2), then φ is a graph isomorphism and graphs Γ1,Γ2 are isomorphic, which we denote by Γ1 ∼= Γ2. If Γ1 = Γ2, then a graph homomorphism/graph isomorphism is a graph endomorphism/automorphism, respectively. A graph Γ is self-complementary if there exists a graph isomorphism σ : Γ → Γ̄, which is referred to as antimorphism or a complementing permutation. Observe that in this case σ is also an antimorphism as a map Γ̄ → Γ. A graph Γ is regular if each vertex has the same number of neighbors. If this number equals k, then we say that it is k-regular. 1Graphs K(5, 2), HGL2(F4), G(5, 2), C5C5 are all isomorphic to the Petersen graph. M. Orel: The core of a vertex-transitive complementary prism 607 If for each pair of vertices u, v ∈ V (Γ) there exists an automorphism φ of Γ such that u = φ(v), then Γ is a vertex-transitive graph. Clearly, each vertex-transitive graph is regular. Lemma 2.1 can be found in [14, Corollaries 2.1.2, 2.1.3], where it is stated in a more general settings. Lemma 2.1. If a graph Γ is vertex-transitive, then α(Γ)ω(Γ) ≤ |V (Γ)|. If the equality holds, then |C ∩ I| = 1 for each clique C and each independent set I that provide the equality. If a graph on n vertices is (n−12 )-regular, then n must be odd. Moreover, it follows from the hand-shaking lemma that n = 4m+ 1 for some integer m ≥ 0. In particular, this is true for all regular self-complementary graphs. By a result of Sachs [37] or Ringel [35], each cycle in an antimorphism of a self-complementary graph has the length divisible by four, except for one cycle of length one, in the case the order of the graph equals 1 modulo 4 (a proof in English can be found in [11, page 12]). Lemma 2.2 is a special case of this fact, and is crucial in the proof of Theorem 3.3. Lemma 2.2. If σ is an antimorphism of a regular self-complementary graph Γ, then there exists a unique vertex v ∈ V (Γ) such that σ(v) = v. A graph Γ is a core if each its endomorphism is an automorphism. Given a graph Γ, we use core(Γ) to denote any subgraph of Γ that is a core and such that there exists some graph homomorphism φ : Γ → core(Γ). Graph core(Γ) is referred to as the core of Γ. It is always an induced subgraph and unique up to isomorphism [15, Lemma 6.2.2]. Clearly, a graph Γ is a core if and only if Γ = core(Γ). On the other hand, core(Γ) is a complete graph if and only if χ(Γ) = ω(Γ). We remark that there always exists a retraction ψ : Γ → core(Γ), i.e. a graph homomorphism that fixes each vertex in core(Γ). In fact, if φ : Γ → core(Γ) is any graph homomorphism, then the restriction φ|V (core(Γ)) is invertible and the composition (φ|V (core(Γ)))−1 ◦ φ is the required retraction. Lemma 2.3 is proved in [39, Theorem 3.2], where it is stated in an old terminology. Its proof can be found also in [15, 17]. Lemma 2.3. If graph Γ is vertex-transitive, then core(Γ) is vertex-transitive. Let Γ be a graph with the vertex set V (Γ) = {v1, . . . , vn}. The complementary prism of Γ is the graph ΓΓ̄, which is obtained from the disjoint union of Γ and its complement Γ̄, by adding an edge between each vertex in Γ and its copy in Γ̄. In this paper we use the following notation. The vertex set of the complementary prism of graph Γ is the set V (ΓΓ̄) =W1 ∪W2, where W1 =W1(ΓΓ̄) = {(v1, 1), . . . , (vn, 1)} and W2 =W2(ΓΓ̄) = {(v1, 2), . . . , (vn, 2)}. The edge set E(ΓΓ̄) is the union of the sets{ {(u, 1), (v, 1)} : {u, v} ∈ E(Γ) } ,{ {(u, 2), (v, 2)} : {u, v} ∈ E(Γ̄) } ,{ {(u, 1), (u, 2)} : u ∈ V (Γ) } . It follows from the definition that a complementary prism ΓΓ̄ is regular if and only if Γ is( n−1 2 ) -regular (see also [7, Theorem 3.6]). The core of a complementary prism for general graph Γ was recently studied in [28] (see also the arXiv version [31]). For regular case, the following result was proved. 608 Ars Math. Contemp. 23 (2023) #P4.07 / 605–613 Lemma 2.4 ([28, Corollary 3.4]). Let Γ be any graph on n vertices that is ( n−1 2 ) -regular. Then one of the following three possibilities is true. (i) Graph ΓΓ̄ is a core. (ii) All vertices of core(ΓΓ̄) are contained in W1, in which case core(ΓΓ̄) ∼= core(Γ). (iii) All vertices of core(ΓΓ̄) are contained in W2, in which case core(ΓΓ̄) ∼= core(Γ̄). The same conclusion can be obtained also if the core of ΓΓ̄ is regular and we exclude some small graphs. Below, K2 is a complete graph on two vertices, and P3 is a path on three vertices. Lemma 2.5 ([28, Corollary 3.6]). Let Γ be any graph, which is not isomorphic to K2, K2, P3, or P3. If core(ΓΓ̄) is regular, then one of the three possibilities in Lemma 2.4 is true. Clearly, Lemma 2.4 is valid for each regular self-complementary graph Γ. The study of such graphs and their vertex-transitive counterparts has origins in the papers [21, 37, 40], which influenced a lot of research related to vertex-transitive self-complementary graphs (see for example [2, 4, 8, 9, 12, 20, 22, 23, 24, 25, 27, 33, 34, 38, 41] and the references therein). In this paper the aim is to to study the core of a complementary prism ΓΓ̄, where Γ is vertex-transitive and self-complementary graph. The following observation is obtained for free, with a double proof. Corollary 2.6. If graph Γ is vertex-transitive and self-complementary graph, then one of the three possibilities in Lemma 2.4 is true. Proof. The claim follows directly from Lemma 2.4. The same claim is deduced also if we combine Lemmas 2.5 and 2.3. In [28] some examples of regular self-complementary graphs Γ are provided such that the statement (ii) or (iii) in Lemma 2.4 is true. In this paper we show that this is not possi- ble for a large class of vertex-transitive self-complementary graphs. It should be mentioned that it was recently proved that a complementary prism ΓΓ̄ is vertex-transitive if and only if Γ is vertex-transitive and self-complementary [31, Corollary 3.8]. Despite our proofs do not rely on this result, it means that this paper studies the core of vertex-transitive comple- mentary prisms. 3 Main results The main result of this paper is Theorem 3.3. Propositions 3.1 and 3.2 are the stepping stones towards its proof. Proposition 3.1. Let Γ be a regular self-complementary graph. If Γ is a core, then ΓΓ̄ is a core. M. Orel: The core of a vertex-transitive complementary prism 609 Proof. Let core(ΓΓ̄) be any core of Γ and let n = |V (Γ)|. Since Γ is ( n−1 2 ) -regular, one of the statements (i), (ii), (iii) in Lemma 2.4 is true. Suppose that (iii) is correct, that is, V (core(ΓΓ̄)) ⊆W2 (3.1) and core(ΓΓ̄) ∼= core(Γ̄). (3.2) Since Γ̄ is a core, (3.2) implies that core(ΓΓ̄) ∼= Γ̄. Hence, (3.1) yields V (core(ΓΓ̄)) =W2. (3.3) Let ψ1(v) = (v, 1), for v ∈ V (Γ), be the canonical isomorphism between Γ and the subgraph of ΓΓ̄, which is induced by the set W1. Similarly, let ψ2(v) = (v, 2), for v ∈ V (Γ), be the canonical isomorphism between Γ̄ and the subgraph induced by W2. If Ψ is any retraction from ΓΓ̄ onto core(ΓΓ̄), and σ is any antimorphism between Γ̄ and Γ, then the composition σ ◦ ψ−12 ◦ (Ψ|W1) ◦ ψ1 is an endomorphism of Γ. Since Γ is a core, the restriction Ψ|W1 is an isomorphism between the subgraphs in ΓΓ̄ that are induced by the sets W1 and W2, respectively. Consequently ψ−12 ◦ (Ψ|W1) ◦ ψ1 : Γ → Γ̄ is an antimorphism. By Lemma 2.2, there exists v ∈ V (Γ) such that ( ψ−12 ◦(Ψ|W1)◦ψ1 ) (v) = v. Consequently, Ψ(v, 1) = (Ψ|W1)(v, 1) = (v, 2). Since Ψ is a retraction, (3.3) implies that Ψ(v, 2) = (v, 2). Since {(v, 1), (v, 2)} is an edge in ΓΓ̄, we have a contradiction. In the same way we see that (ii) in Lemma 2.4 is not possible, which means that ΓΓ̄ is a core. Proposition 3.2. If Γ is a vertex-transitive self-complementary graph on n > 1 vertices, then core(ΓΓ̄) is not a complete graph. Proof. We need to prove that χ(ΓΓ̄) > ω(ΓΓ̄). Since n > 1 and Γ is both self-complement- ary and vertex-transitive, Lemma 2.1 implies that ω(ΓΓ̄) = max{α(Γ), ω(Γ)} = ω(Γ) ≤ √ n. Let I be any independent set in ΓΓ̄. Then I is a disjoint union of some sets I1 ⊆ W1 and I2 ⊆W2. If we write Ii = {(u, i) : u ∈ Ji} for i ∈ {1, 2}, where J1, J2 ⊆ V (Γ), then J1 ∩ J2 = ∅. (3.4) Since J1 and J2 are an independent set and a clique in Γ, respectively, we have |J1| ≤ α(Γ) = ω(Γ) ≤ √ n and |J2| ≤ ω(Γ) = √ n, while Lemma 2.1 and (3.4) imply that |J1| · |J2| < n. Hence, |I| = |J1|+ |J2| < 2 √ n, and therefore χ(ΓΓ̄) ≥ |V (ΓΓ̄)| α(ΓΓ̄) > 2n 2 √ n = √ n ≥ ω(ΓΓ̄). Theorem 3.3. Let Γ be a vertex-transitive self-complementary graph. If Γ is either a core or its core is a complete graph, then ΓΓ̄ is a core. 610 Ars Math. Contemp. 23 (2023) #P4.07 / 605–613 Proof. If Γ is a core, then the claim follows from Proposition 3.1. Hence, we may as- sume that Γ has a complete core and more than one vertex. By Corollary 2.6, one of the statements (i), (ii), (iii) in Lemma 2.4 is true for core(ΓΓ̄). If (ii) or (iii) is correct, then the self-complementarity implies that core(ΓΓ̄) is complete, which contradicts Proposi- tion 3.2. Remark 3.4. The claims in Proposition 3.2 and Theorem 3.3 are not true for some reg- ular self-complementary graphs. In fact, there exists a regular self-complementary graph Γ with a complete core such that core(ΓΓ̄) is complete (see [28, Example 3.5] or [31, Example 5.5]). Recall that a k-regular graph on n vertices is strongly regular with parameters (n, k, λ, µ) if each pair of adjacent vertices has λ common neighbors and each pair of distinct non- adjacent vertices has µ common neighbors. Theorem 3.5 was very recently proved in [28] (see also the arXiv version [31, Theorem 5.7]). The proof relied on application of Lemma 2.4 together with several properties of the Lovász theta function and the graph spectrum. Here we provide a sketch of an alternative proof that essentially copies the proofs in this section and applies a remarkable result that Roberson recently proved [36]. Theorem 3.5 ([28]). If Γ is a strongly regular self-complementary graph, then ΓΓ̄ is a core. Sketch of a proof. To see that ΓΓ̄ does not have a complete core, we copy the proof of Proposition 3.2, where we replace the application of Lemma 2.1 by its analog that can be found in [14, Corolarry 3.8.6 and Theorem 3.8.4] and is valid for all strongly regular graphs. From [36, Corollary 3.6] it follows that Γ is either a core or its core is complete. Then we just copy the proof of Theorem 3.3, where we rely directly on Lemma 2.4 instead on Corollary 2.6. Recall from the introduction that many ‘nice’ graphs are either cores or their cores are complete. Hence it is expected that many vertex-transitive self-complementary graphs fulfil the assumptions in Theorem 3.3. Consequently, we state the following open problem. Problem 3.6. Does there exist a vertex-transitive self-complementary graph Γ such that ΓΓ̄ is not a core? Note that edge-transitive self-complementary graphs are also arc-transitive (see [11]). Since self-complementary graphs are always connected (cf. [11]), it follows that each edge- transitive self-complementary graph is also vertex-transitive. However, such graphs are always strongly regular (cf. [11]) and therefore their complementary prisms are cores by Theorem 3.5. Despite the orders of vertex-transitive self-complementary graphs were fully determined in [27], there is a major gap between the understanding of vertex-transitive self-complementary graphs and the understanding of edge-transitive self-complementary graphs. In fact, the later were completely characterized in [33]. Moreover, the first non- Cayley vertex-transitive self-complementary graph was constructed only in 2001 [24], and the construction is highly nontrivial. We believe that all these facts indicate that Prob- lem 3.6 may be challenging. In a distinct paper [32], we consider the only families of vertex-transitive self-comple- mentary graphs the author is aware of, which are neither cores nor their cores are complete graphs (i.e. they do not satisfy the assumption in Theorem 3.3). Unfortunately they do not solve Problem 3.6. M. Orel: The core of a vertex-transitive complementary prism 611 ORCID iDs Marko Orel https://orcid.org/0000-0002-2544-2986 References [1] A. Alhashim, W. J. Desormeaux and T. W. Haynes, Roman domination in complementary prisms, Australas. J. Comb. 68 (2017), 218–228, https://ajc.maths.uq.edu.au/ pdf/68/ajc_v68_p218.pdf. [2] B. Alspach, J. Morris and V. Vilfred, Self-complementary circulant graphs, Ars Comb. 53 (1999), 187–191. [3] R. M. Barbosa, M. R. Cappelle and E. M. M. Coelho, Maximal independent sets in comple- mentary prism graphs, Ars Comb. 137 (2018), 283–294. [4] R. A. Beezer, Sylow subgraphs in self-complementary vertex transitive graphs, Expo. Math. 24 (2006), 185–194, doi:10.1016/j.exmath.2005.09.003, https://doi.org/10.1016/ j.exmath.2005.09.003. [5] A. E. Brouwer and W. H. Haemers, Spectra of Graphs, Universitext, Springer, New York, 2012. [6] P. J. Cameron and P. A. Kazanidis, Cores of symmetric graphs, J. Aust. Math. Soc. 85 (2008), 145–154, doi:10.1017/S1446788708000815, https://doi.org/10.1017/ S1446788708000815. [7] D. M. Cardoso, P. Carvalho, M. A. A. de Freitas and C. T. M. Vinagre, Spectra, signless Laplacian and Laplacian spectra of complementary prisms of graphs, Linear Algebra Appl. 544 (2018), 325–338, doi:10.1016/j.laa.2018.01.020, https://doi.org/10.1016/j.laa. 2018.01.020. [8] E. Dobson, On self-complementary vertex-transitive graphs of order a product of distinct primes, Ars Comb. 71 (2004), 249–256. [9] E. Dobson and M. Šajna, Almost self-complementary circulant graphs, Discrete Math. 278 (2004), 23–44, doi:10.1016/j.disc.2003.05.002, https://doi.org/10.1016/j.disc. 2003.05.002. [10] M. A. Duarte, L. Penso, D. Rautenbach and U. d. S. Souza, Complexity properties of com- plementary prisms, J. Comb. Optim. 33 (2017), 365–372, doi:10.1007/s10878-015-9968-5, https://doi.org/10.1007/s10878-015-9968-5. [11] A. Farrugia, Self-complementary graphs and generalisations: a comprehensive ref- erence manual, Master’s thesis, University of Malta, Malta, 1999, http://www. alastairfarrugia.net/sc-graph.html. [12] D. Fronček, A. Rosa and J. Širáň, The existence of selfcomplementary circulant graphs, Euro- pean J. Comb. 17 (1996), 625–628, doi:10.1006/eujc.1996.0053, https://doi.org/10. 1006/eujc.1996.0053. [13] I. Garcı́a-Marco and K. Knauer, Beyond symmetry in generalized petersen graphs, 2022, arXiv:2202.06785 [math.CO]. [14] C. Godsil and K. Meagher, Erdős-Ko-Rado Theorems: Algebraic Approaches, volume 149 of Camb. Stud. Adv. Math., Cambridge University Press, Cambridge, 2016, doi:10.1017/ CBO9781316414958, https://doi.org/10.1017/CBO9781316414958. [15] C. Godsil and G. Royle, Algebraic Graph Theory, volume 207 of Grad. Texts Math., Springer, New York, 2001. [16] C. Godsil and G. F. Royle, Cores of geometric graphs, Ann. Comb. 15 (2011), 267–276, doi: 10.1007/s00026-011-0094-5, https://doi.org/10.1007/s00026-011-0094-5. 612 Ars Math. Contemp. 23 (2023) #P4.07 / 605–613 [17] G. Hahn and C. Tardif, Graph homomorphisms: Structure and symmetry, in: Graph Symme- try, Algebraic Methods and Applications. Proceedings of the NATO Advanced Study Institute and séminaire de mathématiques supérieures,Montréal, Canada, Kluwer Academic Publishers, Dordrecht, pp. 107–166, 1997. [18] T. W. Haynes, M. A. Henning, P. J. Slater and L. C. van der Merwe, The complementary product of two graphs, Bull. Inst. Comb. Appl. 51 (2007), 21–30. [19] T. W. Haynes, M. A. Henning and L. C. van der Merwe, Domination and total domination in complementary prisms, J. Comb. Optim. 18 (2009), 23–37, doi:10.1007/s10878-007-9135-8, https://doi.org/10.1007/s10878-007-9135-8. [20] R. Jajcay and C. H. Li, Constructions of self-complementary circulants with no multiplica- tive isomorphisms, European J. Comb. 22 (2001), 1093–1100, doi:10.1006/eujc.2001.0529, https://doi.org/10.1006/eujc.2001.0529. [21] A. Kotzig, Selected open problems in graph theory, in: J. A. Bondy and U. S. R. Murty (eds.), Graph Theory and Related Topics, Academic Press, New York, pp. 358–367, 1979. [22] C. H. Li, On self-complementary vertex-transitive graphs, Comm. Algebra 25 (1997), 3903–3908, doi:10.1080/00927879708826094, https://doi.org/10.1080/ 00927879708826094. [23] C. H. Li, On finite graphs that are self-complementary and vertex-transitive, Aus- tralas. J. Comb. 18 (1998), 147–155, https://ajc.maths.uq.edu.au/pdf/18/ ocr-ajc-v18-p147.pdf. [24] C. H. Li and C. E. Praeger, Self-complementary vertex-transitive graphs need not be Cay- ley graphs, Bull. Lond. Math. Soc. 33 (2001), 653–661, doi:10.1112/s0024609301008505, https://doi.org/10.1112/s0024609301008505. [25] C. H. Li, G. Rao and S. J. Song, On finite self-complementary metacirculants, J. Algebraic Comb. 40 (2014), 1135–1144, doi:10.1007/s10801-014-0522-9, https://doi.org/10. 1007/s10801-014-0522-9. [26] D. Meierling, F. Protti, D. Rautenbach and A. R. de Almeida, Cycles in complementary prisms, Discrete Appl. Math. 193 (2015), 180–186, doi:10.1016/j.dam.2015.04.016, https://doi. org/10.1016/j.dam.2015.04.016. [27] M. Muzychuk, On Sylow subgraphs of vertex-transitive self-complementary graphs, Bull. Lond. Math. Soc. 31 (1999), 531–533, doi:10.1112/S0024609399005925, https://doi. org/10.1112/S0024609399005925. [28] M. Orel, The core of a complementary prism, accepted for publication in Journal of Algebraic Combinatorics. [29] M. Orel, On generalizations of the Petersen graph and the Coxeter graph, Electron. J. Comb. 22 (2015), research paper p4.27, 17 pp., doi:10.37236/3759, https://doi.org/10.37236/ 3759. [30] M. Orel, On Minkowski space and finite geometry, J. Comb. Theory, Ser. A 148 (2017), 145– 182, doi:10.1016/j.jcta.2016.12.004, https://doi.org/10.1016/j.jcta.2016. 12.004. [31] M. Orel, A family of non-Cayley cores based on vertex-transitive or strongly regular self- complementary graphs, 2021, arXiv:2110.10416 [math.CO]. [32] M. Orel, The core of a vertex-transitive complementary prism of a lexicographic product, Art Discrete Appl. Math. (2023), doi:10.26493/2590-9770.1623.7f0, https://doi.org/10. 26493/2590-9770.1623.7f0. M. Orel: The core of a vertex-transitive complementary prism 613 [33] W. Peisert, All self-complementary symmetric graphs, J. Algebra 240 (2001), 209–229, doi: 10.1006/jabr.2000.8714, https://doi.org/10.1006/jabr.2000.8714. [34] S. B. Rao, On regular and strongly-regular self-complementary graphs, Discrete Math. 54 (1985), 73–82, doi:10.1016/0012-365X(85)90063-9, https://doi.org/10.1016/ 0012-365X(85)90063-9. [35] G. Ringel, Selbstkomplementäre Graphen, Arch. Math. (Basel) 14 (1963), 354–358, doi:10. 1007/bf01234967, https://doi.org/10.1007/bf01234967. [36] D. E. Roberson, Homomorphisms of strongly regular graphs, Algebr. Comb. 2 (2019), 481–497, doi:10.5802/alco.50, https://doi.org/10.5802/alco.50. [37] H. Sachs, Über selbstkomplementäre Graphen, Publ. Math. Debrecen 9 (1962), 270–288. [38] D. A. Suprunenko, Self-complementary graphs, Cybern. Syst. Anal. 21 (1985), 559–567, doi: 10.1007/bf01074707, https://doi.org/10.1007/bf01074707. [39] E. Welzl, Symmetric graphs and interpretations, J. Comb. Theory, Ser. B 37 (1984), 235–244, doi:10.1016/0095-8956(84)90056-X, https://doi.org/10.1016/0095-8956(84) 90056-X. [40] B. Zelinka, Self-complementary vertex-transitive undirected graphs, Math. Slovaca 29 (1979), 91–95. [41] H. Zhang, Self-complementary symmetric graphs, J. Graph Theory 16 (1992), 1–5, doi:10. 1002/jgt.3190160102, https://doi.org/10.1002/jgt.3190160102. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.08 / 615–629 https://doi.org/10.26493/1855-3974.2688.2de (Also available at http://amc-journal.eu) Normal Cayley digraphs of dihedral groups with the CI-property* Jin-Hua Xie , Yan-Quan Feng † , Jin-Xin Zhou School of Mathematics and Statistics, Beijing Jiaotong University, Beijing, 100044, P. R. China Received 2 September 2021, accepted 30 January 2023, published online 20 June 2023 Abstract A Cayley (di)graph Cay(G,S) of a group G with respect to a set S ⊆ G is said to be normal if the image of G under its right regular representation is normal in the au- tomorphism group of Cay(G,S), and is called a CI-(di)graph if for every T ⊆ G with Cay(G,S) ∼= Cay(G,T ), there is α ∈ Aut(G) such that Sα = T . A finite group G is called a DCI-group or an NDCI-group if all Cayley digraphs or all normal Cayley digraphs of G are CI-digraphs, respectively, and is called a CI-group or an NCI-group if all Cayley graphs or all normal Cayley graphs of G are CI-graphs, respectively. Motivated by a conjecture proposed by Ádám in 1967, CI-groups and DCI-groups have been actively studied during the last fifty years by many researchers in algebraic graph theory. It took about thirty years to obtain the classification of cyclic CI-groups and DCI- groups, and recently, the first two authors, among others, classified cyclic NCI-groups and NDCI-groups. Even though there are many partial results on dihedral CI-groups and DCI- groups, their classification is still elusive. In this paper, we prove that a dihedral group of order 2n is an NCI-group or an NDCI-group if and only if n = 2, 4 or n is odd. As a direct consequence, we have that if a dihedral group D2n of order 2n is a DCI-group then n = 2 or n is odd-square-free, and that if D2n is a CI-group then n = 2, 9 or n is odd-square- free, throwing some new light on classification of dihedral CI-groups and DCI-groups. As a byproduct, we construct a non-CI Cayley graph of the dihedral group D8, but Holt and Royle in 2020 claimed that D8 is a CI-group by an algorithm there. Keywords: Dihedral group, CI-group, DCI-group, NCI-group, NDCI-group. Math. Subj. Class. (2020): 05C25, 20B25 *The work was supported by the Fundamental Research Funds for the Central Universities (2022JBCG003), the National Natural Science Foundation of China (12071023, 12161141005, 12271024) and the 111 Project of China (B16002). †Corresponding author. E-mail addresses: jinhuaxie@bjtu.edu.cn (Jin-Hua Xie), yqfeng@bjtu.edu.cn (Yan-Quan Feng), jxzhou@bjtu.edu.cn (Jin-Xin Zhou) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 616 Ars Math. Contemp. 23 (2023) #P4.08 / 615–629 1 Introduction Graphs and digraphs considered in this paper are finite and simple, and groups are finite. For a (di)graph Γ, we use V (Γ), E(Γ), Arc(Γ) and Aut(Γ) to denote the vertex set, edge set, arc set, and automorphism group of Γ, respectively, where an arc means a directed edge in a digraph and an ordered pair of adjacent vertices in a graph. Let G be a group and S be a subset of G with 1 ̸∈ S. A digraph with vertex set G and arc set {(g, sg) | g ∈ G, s ∈ S}, denoted by Cay(G,S), is called the Cayley digraph of G with respect to S. If S is inverse-closed, that is, S = S−1 := {x−1 | x ∈ S}, then for two adjacent vertices u and v in Cay(G,S), both (u, v) and (v, u) are arcs, and in this case, we view Cay(G,S) as a graph by identifying the two arcs with one edge {u, v}. Two Cayley (di)graphs Cay(G,S) and Cay(G,T ) are called Cayley isomorphic if there is α ∈ Aut(G) such that Sα = T . Cayley isomorphic Cayley (di)graphs are isomorphic, but the converse is not true. A subset S of G with 1 ̸∈ S is said to be a CI-subset if Cay(G,S) ∼= Cay(G,T ), for some T ⊆ G with 1 ̸∈ T , implies that Cay(G,S) and Cay(G,T ) are Cayley isomorphic. In this case, Cay(G,S) is called a CI-digraph, or a CI-graph when S = S−1. A group G is called a DCI-group if all Cayley digraphs of G are CI-digraphs, and a CI-group if all Cayley graphs of G are CI-graphs. DCI-groups and CI-groups have been widely investigated over last fifty years, and they have been reduced to some special restricted groups in [12, 28]. However, it is still very difficult to determine whether a particular group is a DCI-group or a CI-group; see [2, 5, 11, 20, 26, 31, 32, 37, 38, 39, 40] for examples. In fact, it is even an open problem to classify dihedral DCI-groups or CI-groups. Ádám [1] conjectured that every finite cyclic group is a CI-group. Due to contributions of many researchers like Elspas and Turner [16], Djoković [10], Turner [41], Babai [4], Alspach and Parsons [3], Godsil [22] and Pálfy [33], the cyclic DCI-groups and CI-groups were classified finally by Muzychuk [29, 30]. It follows that a cyclic group of order n is a DCI-group if and only if n = mk where m = 1, 2, 4 and k is odd-square-free, and is a CI-group if and only if either n = 8, 9, 18, or n = mk where m = 1, 2, 4 and k is odd-square-free. The generalised dihedral group Dih(A) over an abelian group A is the group ⟨A, b | b2 = 1, ab = a−1,∀a ∈ A⟩, and in particular, if A is a cyclic group of order n, Dih(A) is the dihedral group D2n of order 2n. There are also some partial results on dihedral DCI-groups or CI-groups. Babai [4] proved that the dihedral group D2p for a prime p is a CI-group. Conder and Li [8] proved that D18 is a CI-group. In [12], it was further proved that D6p (p a prime) is a DCI-group if and only if p ≥ 5, and is a CI-group if and only if p ≥ 3. Recently, Dobson et al. [13] proved that if R is a generalised dihedral CI-group, then for every odd prime p, the Sylow p-subgroup of R has order p, or 9, which reduces dihedral DCI-groups to D2n with n = 2mk (m = 0, 1) and k odd-square-free and dihedral CI-groups to D18 or D2n with n = 2mk (m = 0, 1) and k odd-square-free. Let Cay(G,S) be a Cayley (di)graph of G with respect to a set S ⊆ G. For a given g ∈ G, the right multiplication R(g) : x 7→ xg, x ∈ G, is an automorphism of Cay(G,S), and R(G) := {R(g) | g ∈ G} is a regular group of automorphisms of Cay(G,S), that is, the image of G under its right regular representation. A Cayley (di)graph Cay(G,S) is said to be normal if R(G) is a normal subgroup of Aut(Cay(G,S)). Normality of Cayley (di)graphs is very important because the automorphism groups of normal Cayley (di)graph are actually known; see Godsil [21] or Proposition 2.2. Furthermore, the study of normality of Cayley (di)graphs is currently a hot topic in algebraic graph theory, and we refer to [14, 15, 17, 18, 19, 34, 45, 46] for examples. J.-H. Xie et al.: Normal Cayley digraphs of dihedral groups with the CI-property 617 A group G is called an NDCI-group or an NCI-group if all normal Cayley digraphs or graphs of G are CI-digraphs or CI-graphs, respectively. Obviously, a DCI-group is an NDCI-group and a CI-group is an NCI-group. Li [27] constructed some normal Cayley digraphs of cyclic groups of order 2-powers that are not CI-graphs, and proposed the fol- lowing problem: characterize normal Cayley digraphs which are not CI-digraphs. Similar to DCI-groups and CI-groups, a natural problem is to classify finite NDCI-groups and NCI-groups. Recently, Xie, Feng, Ryabov and Liu [43] classified cyclic NDCI-groups and NCI-groups, and in this paper, we classify dihedral NDCI-groups and NCI-groups. Theorem 1.1. Let n ≥ 2 be an integer and let D2n be the dihedral group of order 2n. Then the following statements are equivalent: (1) D2n is an NDCI-group; (2) D2n is an NCI group; (3) Either n = 2, 4 or n is odd. Li [27, Problem 6.3(3)]) proposed the following problem: are all connected cubic Cay- ley graphs CI-graphs? To prove Theorem 1.1, an infinite family of connected cubic normal non-CI Cayley graphs are constructed in Lemma 4.1, which gives a negative answer to Li’s problem. Classification of NDCI-groups and NCI-groups can be helpful for classification of DCI-groups and CI-groups. In fact, by Theorem 1.1, together with [13], we have the following corollary. Corollary 1.2. If a dihedral group D2n of order 2n is a DCI-group then n = 2 or n is odd-square-free, and if D2n is a CI-group then n = 2, 9 or n is odd-square-free. We believe that the converse of Corollary 1.2 is true. Conjecture 1.3. A dihedral group D2n of order 2n is a DCI-group if and only if n = 2 or n is odd-square-free, and D2n is a CI-group if and only if n = 2, 9 or n is odd-square-free. Note that Holt and Royle [24, 6.2] claimed that D8 is a CI-group, and this is not true by Lemma 4.1, where a non-CI Cayley graph of D8 is constructed and its non-CI-property is also checked by using MAGMA [7]. All the notation and terminologies used in this paper are standard, and for group and graph concepts not defined here, we refer to [6, 9, 35, 36]. 2 Preliminaries In this section, we give some basic concepts and facts that will be needed later. Let F be a free group of rank r and let G be a group generated by a set X of r elements, say X = {x1, x2, . . . , xr}. Then there is a standard free presentation from F to G induced by a bijective mapping from the free generators of F to X . Let G have a defining relation set S, that is, G = ⟨X | S⟩. Let H be a group that can be generated by a set Y of r elements, say Y = {y1, y2, . . . , yr}. The following is the well-known von Dyck’s Theorem [35, 2.2.1]. Proposition 2.1. Assume that all generators yi of H satisfy every relation in S by replacing xi with yi. Then there is an epimorphism from G to H induced by xi 7→ yi for all 1 ≤ i ≤ r. 618 Ars Math. Contemp. 23 (2023) #P4.08 / 615–629 Let Cay(G,S) be a Cayley digraph of a group G with respect to S, and let Aut(G,S) = {α ∈ Aut(G) | Sα = S}. Then Aut(G,S) is a subgroup of Aut(Cay(G,S))1, the stabilizer of 1 in Aut(Cay(G,S)). By Godsil [21], the normalizer of R(G) in Aut(Cay(G,S)) is the semiproduct R(G)⋊Aut(G,S), where R(g)α = R(gα) for all g ∈ G and α ∈ Aut(G,S), and by [44, Propositions 1.3 and 1.5], we have the following. Proposition 2.2. Let Cay(G,S) be a Cayley digraph of a group G with respect to S and let A = Aut(Cay(G,S)). Then NA(R(G)) = R(G)⋊ Aut(G,S) and Cay(G,S) is normal if and only if A1 = Aut(G,S). Babai [4] gave a well-known criterion for a Cayley digraph to be a CI-digraph, that is, a Cayley digraph Cay(G,S) is a CI-digraph if and only if every regular group of automor- phisms of Cay(G,S) isomorphic to G, is conjugate to R(G) in Aut(Cay(G,S)). Based on this, the following proposition is straightforward (also see [27, Corollary 6.9]). Proposition 2.3. Let Cay(G,S) be a normal Cayley digraph of a group G with respect to S. Then Cay(G,S) is a CI-digraph if and only if Aut(Cay(G,S)) has a unique regular subgroup isomorphic to G, that is, R(G). Let G be a finite group and let L ⊆ Aut(G). Write FG(L) = {g | gℓ = g for every ℓ ∈ L}, the fixed-points of L in G. Clearly, FG(L) ≤ G. The following proposition was given in [42, Theorem 2.2], which is about non-normal Cayley digraphs. Proposition 2.4. Let Cay(G,S) be a Cayley digraph. Let 1 ̸= L ≤ Aut(G,S) and H ⊴G be such that for every right coset Hg in G, either L fixes Hg pointwise, or L fixes Hg setwise and is transitive on Hg. Suppose that one of the following holds: (1) |G : FG(L)| > 2; (2) |G : FG(L)| = 2, and there is g ∈ G\FG(L) and h ∈ H such that hg ̸= h−1; (3) |G : FG(L)| = 2, and there is 1 ̸= γ ∈ Aut(G,S) such that FG(⟨γ⟩) ̸= FG(L) and γ fixes every coset of H in G setwise. Then Cay(G,S) is non-normal. 3 Automorphisms and holomorphs of dihedral groups In this section we collect some details about automorphism groups and holomorphs of dihedral groups, which are crucial for the proof of Theorem 1.1. Let G be a finite group and let g ∈ G. Denote by o(g) the order of g in G. Let p be a prime and π a set of primes. Denote by Gp a Sylow p-subgroup of G. An element g of G is called a π-element if all prime factors of o(g) belong to π, and a p′-element if o(g) has no factor p. If G is soluble, denote by Gπ a Hall π-subgroup of G, and by Gp′ a Hall p′-subgroup of G. Let n be a positive integer. We first make the following convention: n = ∏m i=1 prii ≥ 2, Cn = Cpr11 ×· · ·×Cprmm = ⟨a1⟩×· · ·×⟨am⟩, a = a1 . . . am, (3.1) where p1, . . . , pm are all distinct prime factors of n, Cprii = ⟨ai⟩ ∼= Zprii for each 1 ≤ i ≤ m, and Cn = ⟨a⟩ ∼= Zn. By [36, Theorem 7.3], we have Aut(Cn) = Aut(Cpr11 )× Aut(Cpr22 )× · · · × Aut(Cprmm ), (3.2) J.-H. Xie et al.: Normal Cayley digraphs of dihedral groups with the CI-property 619 where Aut(Cprii ) is viewed as the subgroup of Aut(Cn) by identifying αi ∈ Aut(Cprii ) as the automorphism of Cn induced by ai 7→ aαii and aj 7→ aj for all j ̸= i. Further we set D2n = ⟨a, b | an = b2 = 1, bab = a−1⟩ = Cn ⋊ ⟨b⟩. (3.3) By Proposition 2.1, for ai ∈ ⟨a⟩ with i ∈ Zn, there is an automorphism of D2n, denoted by θai , which is induced by θai : a 7→ a, b 7→ bai. (3.4) Then o(θai) = o(ai) and ⟨θa⟩ ∼= Zn. Write ai = a1 . . . ai−1ai+1 . . . am. Then o(ai) = n/prii , and hence ⟨θa⟩ = ⟨θai⟩ × ⟨θai⟩. In fact, ⟨θai⟩ and ⟨θai⟩ are the unique Sylow pi-subgroup and the unique Hall p′i-subgroup of ⟨θa⟩, respectively. Clearly, θaiθaj = θai+j and θkai = θaik for all i, j, k ∈ Zn. (3.5) Again by Propostion 2.1, we also view Aut(Cn) as the subgroup of Aut(D2n) by iden- tifying β ∈ Aut(Cn) as the automorphism of Aut(D2n) induced by a 7→ aβ and b 7→ b. Then for each 1 ≤ i ≤ m, we have Aut(Cprii ) ≤ Aut(D2n). Lemma 3.1. Let n ≥ 3. Then Aut(D2n) has the following properties. (1) Aut(D2n) = ⟨θa⟩⋊Aut(Cn), and ⟨θa⟩ is the kernel of the natural action of Aut(D2n) on Cn. Furthermore, θβx = θxβ for all x ∈ ⟨a⟩ and β ∈ Aut(Cn); (2) For every 1 ≤ i ≤ m, ⟨θa⟩ ⋊ Aut(Cprii ) = (⟨θai⟩ ⋊ Aut(Cprii )) × ⟨θai⟩, where ai = a1 . . . ai−1ai+1 . . . am; (3) Assume p1 > p2 > · · · > pm. For every 1 ≤ i ≤ m, set πi = {p1, . . . , pi}. Then Aut(D2n) has a normal Hall πi-subgroup and Aut(D2n)πi = ⟨θa1...ai⟩⋊Aut(Cn)πi , where Aut(Cn)πi = Aut(Cpr11 )πi × · · · × Aut(Cprii )πi . Proof. It is easy to prove that Aut(D2n) = ⟨θa⟩ ⋊ Aut(Cn) (also see [25, Theorem 7.2]). Let K be the kernel of Aut(D2n) acting on Cn. Then aγ = a for all γ ∈ K, which implies ⟨θa⟩ ≤ K. Furthermore, K is transitive on b⟨a⟩ as ⟨θa⟩ is transitive. Since D2n = ⟨a, b⟩ and K ≤ Aut(D2n), we have Kb = 1, implying that K is regular on b⟨a⟩. It follows that |K| = |b⟨a⟩| = n, and hence K = ⟨θa⟩. For β ∈ Aut(Cn), we have bβ = b, and for x ∈ ⟨a⟩, θx fixes ⟨a⟩ pointwise. Since β fixes ⟨a⟩ setwise, aθx β = (aβ −1 )θxβ = aβ −1β = a = aθxβ , and bθx β = (bβ −1 )θxβ = bθxβ = (bx)β = bxβ = bθxβ . Since D2n = ⟨a, b⟩, we obtain θβx = θxβ . This completes the proof of part (1). Recall that ⟨θa⟩ = ⟨θai⟩ × ⟨θai⟩. Let β ∈ Aut(Cprii ). Since ai = a1 . . . ai−1ai+1 . . . am, we have a β i = ai, and θ β ai = θ(ai)β = θai , that is, θaiβ = βθai . Thus, ⟨θa⟩⋊ Aut(Cprii ) = (⟨θai⟩⋊ Aut(Cprii ))× ⟨θai⟩. This completes the proof of part (2). 620 Ars Math. Contemp. 23 (2023) #P4.08 / 615–629 Let 1 ≤ i ≤ m. Note that Aut(Cprii ) ∼= Zpri−1i (pi−1) if pi is odd, and Aut(Cp ri i ) ∼= Zpi × Zpri−2i if pi = 2 and ri ≥ 2. Since p1 > p2 > · · · > pm, we have Aut(Cp rk k )πi = 1 for i < k ≤ m, and hence Aut(Cn)πi = Aut(Cpr11 )πi × · · · × Aut(Cprii )πi . It is easy to see that θa1...ai = θa1 . . . θai has order p r1 1 . . . p ri i . Then ⟨θa1...ai⟩ is a Hall πi-subgroup of ⟨θa⟩, and hence ⟨θa1...ai⟩ is characteristic in ⟨θa⟩. Since ⟨θa⟩ ⊴ Aut(D2n), we have ⟨θa1...ai⟩⊴Aut(D2n), and ⟨θa1...ai⟩⋊Aut(Cn)πi is a Hall πi-subgroup of Aut(D2n). Recall that Aut(D2n) = ⟨θa⟩⋊Aut(Cn). To prove ⟨θa1...ai⟩⋊Aut(Cn)πi⊴Aut(D2n), it suffices to show that Aut(Cn)θaπi ≤ ⟨θa1...ai⟩⋊Aut(Cn)πi , or alternatively, Aut(Cprjj ) θa πi ≤ ⟨θa1...ai⟩ ⋊ Aut(Cn)πi for every 1 ≤ j ≤ i. Now take α ∈ Aut(Cprjj )πi . Then a α k = ak for every 1 ≤ k ≤ m with k ̸= j, and aαj ∈ ⟨aj⟩. It follows that αθa = θ−1a αθa = θ −1 a θ α−1 a α = θa−1θaα−1α = θa−1aα−1α = θ a−1j a α−1 j α ∈ ⟨θaj ⟩Aut(Cprjj )πi , where a−1aα −1 = a−11 a −1 2 . . . a −1 m a1 . . . aj−1a α−1 j aj+1 . . . am = a −1 j a α−1 j ∈ ⟨aj⟩. Since 1 ≤ j ≤ i, we have ⟨θaj ⟩Aut(Cprjj )πi ≤ ⟨θa1...ai⟩⋊ Aut(Cn)πi , and hence Aut(C p rj j )θaπi ≤ ⟨θa1...ai⟩ ⋊ Aut(Cn)πi , as required. Thus, Aut(D2n) has a normal Hall πi-subgroup, that is, Aut(D2n)πi = ⟨θa1...ai⟩⋊Aut(Cn)πi . This complete the proof of part (3). For a finite group G, the right regular representation R(G) and the automorphism group Aut(G) are permutation groups on G. Furthermore, R(G) ∩ Aut(G) = 1, and R(G)Aut(G) = R(G) ⋊ Aut(G), where R(g)α = R(gα) for all g ∈ G and α ∈ Aut(G). The normalizer of R(G) in the symmetric group SG on G is called the holomorph of G, denoted by Hol(G), and by [36, Lemma 7.16], Hol(G) = R(G)⋊ Aut(G). Now we have Hol(D2n) = R(D2n)⋊ Aut(D2n) = R(D2n)⋊ (⟨θa⟩⋊ Aut(Cn)). (3.6) Lemma 3.2. Let n be odd. Using the notations and formulae in Equations (3.1) – (3.6), we have the following. (1) For all d ∈ D2n and α ∈ Aut(D2n), ⟨R(a)⟩⟨θa⟩ = ⟨R(a)⟩ × ⟨θa⟩⊴ Hol(D2n); (2) Assume p1 > p2 > · · · > pm. For each 1 ≤ i ≤ m, set πi = {p1, . . . , pi}. Then Hol(D2n)πi ≤ (⟨R(a)⟩ × ⟨θa⟩) ⋊ Aut(Cn)πi ⊴ Hol(D2n), where Aut(Cn)πi = Aut(Cpr11 )πi × · · · × Aut(Cprii )πi . Furthermore, ⟨R(b)⟩Aut(Cn)2 = ⟨R(b)⟩ × Aut(Cn)2 is a Sylow 2-subgroup of Hol(D2n), where Aut(Cn)2 = Aut(Cpr11 )2 × Aut(Cpr22 )2 × · · · × Aut(Cprmm )2; (3) Let 1 ≤ i, j ≤ m with i ̸= j. Then Aut(Cprii ), under conjugacy, fixes each pj- element in ⟨R(a)⟩ × ⟨θa⟩, and if α ∈ Aut(Cprii ) fixes an element of order p ri i in ⟨R(a)⟩ × ⟨θa⟩, then α = 1. J.-H. Xie et al.: Normal Cayley digraphs of dihedral groups with the CI-property 621 Proof. Recall that R(d)α = R(dα) for all d ∈ D2n and α ∈ Aut(D2n). Then R(a)θa = R(aθa) = R(a), that is, R(a) commutes with θa. Since ⟨R(a)⟩ ∩ ⟨θa⟩ = 1, we have ⟨R(a)⟩⟨θa⟩ = ⟨R(a)⟩ × ⟨θa⟩. Since n is odd, ⟨R(a)⟩ × ⟨θa⟩ is a Hall 2′-subgroup of R(D2n) ⋊ ⟨θa⟩, and hence ⟨R(a)⟩ × ⟨θa⟩ is characteristic in R(D2n) ⋊ ⟨θa⟩. It follows that ⟨R(a)⟩ × ⟨θa⟩ ⊴ Hol(D2n), as R(D2n) ⋊ ⟨θa⟩ ⊴ Hol(D2n) by Equation (3.6). This completes the proof of part (1). Now we prove part (2). By part (1), ⟨R(a)⟩× ⟨θa⟩⊴Hol(D2n), and by Equation (3.6), Hol(D2n) = R(D2n)⋊ (⟨θa⟩⋊ Aut(Cn)) = (R(D2n)⋊ ⟨θa⟩)⋊ Aut(Cn). Write A = ⟨R(a)⟩ × ⟨θa⟩. Then A ∩ Aut(Cn) = 1, and by Equation (3.6), (R(D2n) ⋊ ⟨θa⟩)/A is a normal subgroup of order 2 in Hol(D2n)/A, and hence lies in the center of Hol(D2n)/A. Thus, Hol(D2n)/A = (R(D2n)⋊ ⟨θa⟩)/A× Aut(Cn)A/A, where Aut(Cn)A/A ∼= Aut(Cn). Since Aut(Cn) is abelian, Hol(D2n)/A is abelian, and therefore, (⟨R(a)⟩ × ⟨θa⟩)⋊ Aut(Cn)πi = A⋊ Aut(Cn)πi ⊴ Hol(D2n). Since n is odd, all pi are odd and hence 2 ̸∈ πi for each 1 ≤ i ≤ m. Clearly, Hol(D2n)/((R(D2n)⋊ ⟨θa⟩)⋊ Aut(Cn)πi) ∼= Aut(Cn)/Aut(Cn)πi . Then for every Hall πi-subgroup Hol(D2n)πi , Hol(D2n)πi ≤ (R(D2n) ⋊ ⟨θa⟩) ⋊ Aut(Cn)πi , and it follows that Hol(D2n)πi ≤ (⟨R(a)⟩×⟨θa⟩)⋊Aut(Cn)πi , as |(R(D2n)⋊ ⟨θa⟩) : (⟨R(a)⟩ × ⟨θa⟩)| = 2. By Lemma 3.1(3), Aut(Cn)πi = Aut(Cpr11 )πi × · · · × Aut(Cprii )πi . Since Aut(Cn) fixes b, we have ⟨R(b)⟩Aut(Cn) = ⟨R(b)⟩ × Aut(Cn), and therefore, ⟨R(b)⟩Aut(Cn)2 = ⟨R(b)⟩ × Aut(Cn)2. By Equation (3.6), |Hol(D2n)2| = 2|Aut(Cn)2|, so ⟨R(b)⟩ × Aut(Cn)2 is a Sylow 2-subgroup of Hol(D2n). Clearly, Aut(Cn)2 = Aut(Cpr11 )2 × Aut(Cpr22 )2 × · · · × Aut(Cprmm )2. This completes the proof of part (2). To prove part (3), let 1 ≤ i, j ≤ m with j ̸= i. Recall that pi is odd. By part (1), ⟨R(a)⟩ × ⟨θa⟩⊴ Hol(D2n). Let α ∈ Aut(Cprii ) and let x be a pj-element in ⟨R(a)⟩ × ⟨θa⟩. Since ⟨R(aj)⟩ × ⟨θaj ⟩ is a normal Sylow pj-subgroup of ⟨R(a)⟩ × ⟨θa⟩, we may take x = R(aj)sθtaj for some s, t ∈ Z p rj j . Since j ̸= i, aαj = aj and hence xα = (R(aj) sθtaj ) α = R(aαj ) sθtaαj = R(aj) sθtaj = x. Thus, Aut(Cprir ) fixes each pj-element in ⟨R(a)⟩ × ⟨θa⟩. Now assume that α ∈ Aut(Cprii ) fixes an element of order p ri i in ⟨R(a)⟩ × ⟨θa⟩. Since ⟨R(ai)⟩ × ⟨θai⟩ is a normal Sylow pi-subgroup of ⟨R(a)⟩ × ⟨θa⟩, α fixes an element of order prii in ⟨R(ai)⟩ × ⟨θai⟩, say R(ai)kθℓai for some k, ℓ ∈ Zprii . Since o(R(ai)) = o(θai) = p ri i , we have (k, pi) = 1 or (ℓ, pi) = 1. Furthermore, R(aki )θaℓi = R(ai) kθℓai = (R(ai) kθℓai) α = R((aki ) α)θ(aℓi)α , and since ⟨R(a)⟩ ∩ ⟨θa⟩ = 1, we have (aki )α = aki and (aℓi)α = aℓi , which implies that (ai) α = ai as (k, pi) = 1 or (ℓ, pi) = 1. Thus, α = 1, completing the proof of part (3). 622 Ars Math. Contemp. 23 (2023) #P4.08 / 615–629 4 Proof of Theorem 1.1 First we construct normal Cayley graphs on dihedral groups which are not CI-graphs. Re- call that D2n = ⟨a, b | an = b2 = 1, bab = a−1⟩, as given in Equation (3.3). Lemma 4.1. Let Γ = Cay(D2n, {a, a−1, b}). Then we have the following. (1) If n = 4 then Γ is non-normal, and if n ≥ 5 then Γ is normal; (2) If n ≥ 4 is even, then Γ is not a CI-graph. Proof. Let n ≥ 4. Write A = Aut(Γ) and S = {a, a−1, b}. It is easy to see that Aut(D2n, S) = ⟨α⟩, where α is the automorphism of D2n induced by a 7→ a−1 and b 7→ b. Since S = S−1 and ⟨S⟩ = D2n, Γ is a connected cubic graph of order 2n. Clearly, Γ ∼= Cn ×K2 is the ladder graph of order 2n, where Cn is the cycle of length n and K2 is the complete graph of order 2 (the Cartesian product Γ1×Γ2 of two graphs Γ1 and Γ2 have vertex set {(u1, u2) | ui ∈ V (Γi), i = 1, 2} and edge set {{(u1, u2), (v1, v2)} | either u1 = v1 and (u2, v2) ∈ E(Γ2), or u2 = v2 and (u1, v1) ∈ E(Γ1)}). Note that C4 ∼= K2 ×K2, and for n ≥ 5, Cn and K2 are relatively prime, that is, Cn cannot be a Cartesian product of K2 and a graph because Cn contains no cycle of length 4 (see [23, Lemma 6.3]). Assume n = 4. Then Γ ∼= C4 ×K2 ∼= K2 ×K2 ×K2 ∼= K4,4 − 4K2, the complete bipartite graph K4,4 minus one factor. Then A ∼= S4 × C2, where S4 is the symmetric group of degree 4. Since A ̸= R(D8)⋊ Aut(D8, S) = R(D8)⋊ ⟨α⟩, Γ is non-normal. Assume that n ≥ 5. Since Cn and K2 are relatively prime, by [23, Corollary 6.12] we have A ∼= Aut(Cn) × Aut(K2) ∼= D2n × Z2. It follows that |A : R(D2n)| = 2, and so R(D2n)⊴A, that is, Γ is a normal Cayley graph. This completes the proof of part (1). To prove that Γ is not a CI-graph for n ≥ 4 and n even, by the well-known Babai criterion (see [4]), we only need to show that A has a regular dihedral subgroup, which is not conjugate to R(D2n) in A. Note that R(a)R(b) = R(a−1), R(b)α = R(b) and R(a)α = R(a−1). Thus, R(b)α is an involution and R(a)(R(b)α) = R(a−1)α = R(a), that is, R(a) commutes with R(b)α. Since R(a) has order n and n is even, R(ab)α = R(a)(R(b)α) has order n. Furthermore, (R(ab)α)R(b) = R(ab)R(b)α = R(ba)α = αR(ba)α = αR(ba−1) = (R(ab)α)−1. Thus, ⟨R(ab)α,R(b)⟩ is a dihedral group of order 2n. Note that (R(ab)α)2 = R(ab)αR(ab)α = R(ab)R(ab)α = R(ab)R(a−1b) = R(a2). Clearly, R(a2) has order n/2 and ⟨R(a2)⟩ ⊴ ⟨R(ab)α,R(b)⟩. Since ⟨R(a)⟩ is semireg- ular on D2n with two orbits, ⟨R(a2)⟩ has four orbits on D2n, that is, ⟨a2⟩, a⟨a2⟩, b⟨a2⟩ and ba⟨a2⟩. The involution R(b) interchanges ⟨a2⟩ and b⟨a2⟩, and a⟨a2⟩ and ba⟨a2⟩. Fur- thermore, R(ab)α interchanges ⟨a2⟩ and ba⟨a2⟩, and a⟨a2⟩ and b⟨a2⟩. It follows that ⟨R(ab)α,R(b)⟩ is transitive on D2n, and hence regular as |⟨R(ab)α,R(b)⟩| = 2n. By MAGMA [7], ⟨R(ab)α,R(b)⟩ is not conjugate to R(D8) for n = 4, and so Γ is not a CI-graph. Let n ≥ 5. Suppose Γ is a CI-graph. By part (1), Γ is normal, and by Proposition 2.3, R(D2n) = ⟨R(ab)α,R(b)⟩, forcing α ∈ R(D2n), a contradiction. Thus, Γ is not a CI-graph. Now we are ready to prove Theorem 1.1. J.-H. Xie et al.: Normal Cayley digraphs of dihedral groups with the CI-property 623 Proof of Theorem 1.1. Let n ≥ 2. First we prove (1) and (3) are equivalent, that is, D2n is an NDCI-group if and only if either n = 2, 4 or n is odd. The necessity follows from Lemma 4.1. To prove the sufficiency, let n be odd, or n = 2, 4, and we only need to prove that D2n is an NDCI-group. Let Γ = Cay(D2n, S) be a normal Cayley digraph. It suf- fices to show that Γ is a CI-digraph. Note that we use the notations or formulae from Equations (3.1) – (3.6). Let A = Aut(Γ). By Proposition 2.2, A = R(D2n)Aut(D2n, S) ≤ Hol(D2n) and A1 = Aut(D2n, S) ≤ Aut(D2n). Assume n = 2. Then D4 ∼= C2 × C2, and D4 is an NDCI-group because C2 × C2 is a DCI-group. Assume n = 4. By Lemma 4.1, D8 is a non-DCI-group. However, with the help of MAGMA [7], one may easily check that D8 is an NDCI-group; it also can be proved by restricting the valency of Γ to be no more than 3 because the complement of Γ is isomorphic to Cay(D8,D8\(S ∪ {1})) and by using the fact |A| is a divisor of |Hol(D8)| = 64. Assume that n is odd. Without loss of generality, we may further assume that p1 > p2 > · · · > pm, where pi’s are all prime factors of n. For 1 ≤ i ≤ m, we may set πi = {p1, p2, . . . , pi}. Recall that A = R(D2n)Aut(D2n, S) ≤ Hol(D2n) and A1 = Aut(D2n, S) ≤ Aut(D2n). Since Cn is characteristic in D2n, A1 fixes Cn setwise. By Lemma 3.1(1), ⟨θa⟩ is the kernel of Aut(D2n) on Cn, and therefore, the kernel of A1 on Cn is ⟨θa⟩ ∩ A1. Then A1/(⟨θa⟩ ∩ A1) induces an subgroup of Aut(Cn), say B, and |A1| = |⟨θa⟩ ∩ A1| · |B|. Note that Aut(Cn) is viewed as a subgroup of Aut(D2n), and so is B, too. Then A1 ≤ ⟨⟨θa⟩ ∩ A1, B⟩. On the other hand, ⟨θa⟩ ∩ A1 is characteristic in ⟨θa⟩, and hence normal in Aut(D2n), forcing that (⟨θa⟩ ∩ A1)B = (⟨θa⟩ ∩ A1) ⋊ B. It follows that ⟨⟨θa⟩ ∩ A1, B⟩ = (⟨θa⟩ ∩ A1)⋊ B, and since |A1| = |(⟨θa⟩ ∩ A1)| · |B|, we have A1 = (⟨θa⟩ ∩A1)⋊B. Thus, A = R(D2n)⋊A1, A1 = (⟨θa⟩ ∩A1)⋊B with B ≤ Aut(Cn). Let G be a regular subgroup of A such that G ∼= D2n. To prove that Γ is a CI-digraph, by Proposition 2.3 it suffices to show that G = R(D2n). We argue by contradiction, and we suppose that G ̸= R(D2n). By Lemma 3.2(2), ⟨R(b)⟩ × Aut(Cn)2 = ⟨R(b)⟩ × Aut(Cpr11 )2 × Aut(Cpr22 )2 × · · · × Aut(Cprmm )2 is a Sylow 2-subgroup of Hol(D2n), denote by HD2. Let B2 be a Sylow 2-subgroup of B. Then we have B2 ≤ Aut(Cn)2, and hence ⟨R(b)⟩ ×B2 is a Sylow 2-subgroup of A. Since Sylow 2-subgroups of A are conjugate by the second Sylow theorem and |G2| = 2, there is d ∈ A such that G∩ (⟨R(b)⟩×B2)d ̸= 1. Thus, Gd −1 ∩ (⟨R(b)⟩ × B2) ̸= 1. Write H = Gd −1 . Then H ≤ A is regular on V (Γ), and since R(D2n)⊴A and R(D2n) ̸= G, we have H ̸= R(D2n) and |H ∩HD2| = 2. By Equation (3.6) and Lemma 3.2(1), Hol(D2n) = R(D2n)⋊ (⟨θa⟩⋊ Aut(Cn)) = ((⟨R(a)⟩ × ⟨θa⟩)⋊ ⟨R(b)⟩)⋊ Aut(Cn). Since H ∼= D2n, we may assume that H = ⟨v, w⟩ ∼= D2n, o(v) = n, o(w) = 2, vw = v−1, and w ∈ HD2. 624 Ars Math. Contemp. 23 (2023) #P4.08 / 615–629 Claim 1. v ∈ ⟨R(a)⟩ × ⟨θa⟩. Proof of Claim 1. Since o(v) = n, we have o(vn/p ri i ) = prii . Write T0 = {1}, T1 = {1, vn/p r1 1 }, T2 = {1, vn/p r1 1 , vn/p r2 2 }, . . ., Tm = {1, vn/p r1 1 , vn/p r2 2 , . . . , vn/p rm m }. Clearly, ⟨v⟩ = ⟨Tm⟩. To finish the proof of Claim 1, it suffices to show that Tm ⊆ ⟨R(a)⟩ × ⟨θa⟩. To do this, we proceed by induction on k to show Tk ⊆ ⟨R(a)⟩ × ⟨θa⟩, where 0 ≤ k ≤ m. Clearly, T0 ⊆ ⟨R(a)⟩ × ⟨θa⟩, and we may let k > 0. By induction hypothesis, we may assume that Tj ⊆ ⟨R(a)⟩ × ⟨θa⟩ for all 0 ≤ j < k and aim to show Tk ⊆ ⟨R(a)⟩ × ⟨θa⟩. Since Tk = Tk−1 ∪ {vn/p rk k }, we only need to show that vn/p rk k ∈ ⟨R(a)⟩ × ⟨θa⟩. Since o(vn/p rk k ) = prkk , v n/p rk k is a πk-element. By Lemma 3.2(2), vn/p rk k ∈ (⟨R(a)⟩× ⟨θa⟩)⋊ (Aut(Cpr11 )πk × · · · × Aut(Cprkk )πk). Then we may write v n/p rk k = xβ1β2 . . . βk, where x ∈ ⟨R(a)⟩ × ⟨θa⟩ and βj ∈ Aut(Cprjj )πk for 1 ≤ j ≤ k. Clearly, v commutes with every element in Tk−1, and so does vn/p rk k . Since ⟨R(a)⟩ × ⟨θa⟩ is abelian, x commutes with every element in Tk−1 because Tk−1 ⊆ ⟨R(a)⟩ × ⟨θa⟩. For every 1 ≤ ℓ ≤ k − 1, by Lemma 3.2(3) we have that if j ̸= ℓ then βj ∈ Aut(Cprjj )πk commutes with every element of order prℓℓ in Tk−1, and then βℓ commutes with every element of order prℓℓ in Tk−1 because βℓ = β −1 ℓ−1 . . . β −1 1 x −1vn/p rk k β−1k . . . β −1 ℓ+1, which implies that βℓ commutes with an element of order prℓℓ in ⟨R(a)⟩×⟨θa⟩ as Tk−1 ⊆ ⟨R(a)⟩× ⟨θa⟩. Again by Lemma 3.2(3), we obtain βℓ = 1. It follows vn/p rk k = xβk = R(y)θzβk for some y, z ∈ ⟨a⟩ and βk ∈ Aut(Cprkk )πk . Since R(y) ∈ A, we have θzβk ∈ A, so θzβk ∈ A1 = (⟨θa⟩ ∩ A1) ⋊ B, where B ≤ Aut(Cn). Then there exist θz′ ∈ ⟨θa⟩ ∩ A1 and βk′ ∈ B such that θzβk = θz′βk′ . It follows that θ−1z′ θz = βk′β −1 k ∈ ⟨θa⟩ ∩ Aut(Cn) = 1, that is, θz = θz′ ∈ ⟨θa⟩ ∩ A1 ≤ A1 and βk = βk′ ∈ B ≤ A1. Suppose βk ̸= 1. Recall that p1 > p2 > · · · > pk, πk = {p1, p2, . . . , pk} and Aut(Cprkk ) ∼= Z p rk−1 k (pk−1) . Then every πk-element in Aut(Cprkk ) is a pk-element, and since βk ∈ Aut(Cprkk )πk , o(βk) has order pk-power. Let β be the automorphism of D2n induced by β : ak 7→ a p rk−1 k +1 k , b 7→ b, ai 7→ ai for every i ̸= k. Let L = ⟨β⟩. Then L is the unique subgroup of order pk of ⟨βk⟩. It follows that rk ≥ 2 and L ≤ ⟨βk⟩ ≤ A1 = Aut(D2n, S). Write M = ⟨a1, . . . , ak−1, apkk , ak+1, . . . , am⟩, and K = ⟨a p rk−1 k k ⟩. Clearly, 1 < K ⊴ D2n. Note that every element of D2n has the form (a1a2 · · · am)t or b(a1a2 · · · am)t for some t ∈ Zn. It is easy to see that the set of fixed-points of L in D2n is M ∪ bM , that is, FD2n(L) = M ∪ bM . Furthermore, L is transitive on Kc for every c ∈ D2n\(M ∪ bM), and |D2n : FD2n(L)| ≥ pk > 2 as pk is odd. By Proposition 2.4(1), Γ is non-normal, a contradiction. Then βk = 1. It follows that vn/p rk k = R(y)θzβk = R(y)θz ∈ ⟨R(a)⟩ × ⟨θa⟩ and Tk ⊆ ⟨R(a)⟩ × ⟨θa⟩, as required. This completes the proof of Claim 1. By Claim 1, we may assume that v = R(a)kθℓa for some k, ℓ ∈ Zn. Since θa fixes ⟨a⟩ pointwise, we have 1⟨v⟩ = ⟨ak⟩, the orbit of ⟨v⟩ containing 1 in D2n, and since ⟨v⟩ ≤ H is J.-H. Xie et al.: Normal Cayley digraphs of dihedral groups with the CI-property 625 semiregular, |1⟨v⟩| = |⟨ak⟩| = o(v) = n, forcing o(ak) = n. Thus, ⟨ak⟩ = ⟨a⟩ and hence (k, n) = 1. Since v ∈ H , if necessary we replace v by a power of v, and then one may let v = R(a)θℓa for some ℓ ∈ Zn. Recall that w ∈ H with o(w) = 2 and w ∈ HD2 = ⟨R(b)⟩ × Aut(Cn)2. If w ∈ Aut(Cn)2 = Aut(Cpr11 )2 × Aut(Cpr22 )2 × · · · × Aut(Cprmm )2, then w fixes 1, contradicting the regularity of H . Thus, w ∈ R(b)Aut(Cn)2, and by Lemma 3.2(2), we have w = R(b)ε1ε2 . . . εm, where εi ∈ Aut(Cprii )2 and ε 2 i = 1 for every 1 ≤ i ≤ m. Claim 2. For every 1 ≤ k ≤ m, either prkk | ℓ and εk = 1, or (ℓ, pk) = 1 and εk ̸= 1. Proof of Claim 2. Write ε = ε1 . . . εm ∈ Aut(Cn). Since H ∼= D2n, we have vR(b)ε = vw = v−1 = R(a−1)θa−ℓ . By Lemma 3.2(1), we get θR(b)a = θaR(b) θaR(b) = θaR(b θab) = θaR(a −1) and (θℓa) R(b) = θℓaR(a −ℓ) = R(a−ℓ)θaℓ . Since vR(b) = (v−1)ε = R((aε)−1)θ(aε)−ℓ , we have R(a−(ℓ+1))θaℓ = R(a −1)R(a−ℓ)θaℓ = R(a) R(b)(θℓa) R(b) = vR(b) = R((aε)−1)θ(aε)−ℓ . Since ⟨R(a)⟩ ∩ ⟨θa⟩ = 1, we deduce R(aℓ+1) = R(aε) and θaℓ = θ(aε)−ℓ . It follows that aε = aℓ+1 and (aε)ℓ = a−ℓ. This yields aℓ(ℓ+2) = 1, so n | ℓ(ℓ + 2) and prkk | ℓ(ℓ + 2) for every 1 ≤ k ≤ m. If pk | ℓ and pk | ℓ + 2, then pk | 2, which is impossible because n is odd. It follows that either prkk | ℓ and (pk, ℓ+ 2) = 1, or p rk k | (ℓ+ 2) and (pk, ℓ) = 1. Assume prkk | ℓ and (pk, ℓ+ 2) = 1. Since aε = aℓ+1, we have aε = (a1a2 . . . am) ε = aε11 a ε2 2 . . . a εm m = a ℓ+1 = aℓ+11 · a ℓ+1 k−1aka ℓ+1 k+1 . . . a ℓ+1 m , implying aεkk = ak, that is, εk = 1. Assume prkk | (ℓ + 2) and (pk, ℓ) = 1. Since (aε)ℓ = a−ℓ, we have (aℓk)εk = (aℓk)−1, and so εk ̸= 1 because (pk, ℓ) = 1 implies o(aℓk) = p rk k . This completes the proof of Claim 2. If prkk | ℓ for all 1 ≤ k ≤ m, by Claim 2 we have εk = 1, and hence v = R(a)θℓa = R(a) and w = R(b)ε1ε2 . . . εm = R(b), yielding H = R(D2n), a contradiction. Thus, εk ̸= 1 for some k. To simplify notation, from now on we do not assume p1 > p2 > · · · > pm any more, which has no confusion. Then we may assume that there exists 1 ≤ s ≤ m such that (ℓ, p1 . . . ps) = 1, εj ̸= 1 for all 1 ≤ j ≤ s, prs+1s+1 . . . prmm | ℓ, and εj = 1 for all s + 1 ≤ j ≤ m. It follows that o(aℓ) = pr11 . . . prss , forcing ⟨aℓ⟩ = ⟨a1a2 . . . as⟩ and ⟨θℓa⟩ = ⟨θa1a2...as⟩. Let L = ⟨θa1a2...as⟩. Recall that v = R(a)θℓa ∈ H ≤ A. It follows from R(D2n) ≤ A that θℓa ∈ A, implying θℓa ∈ A1 = Aut(D2n, S). Then L = ⟨θℓa⟩ ≤ Aut(D2n, S). Let K = ⟨a1a2 . . . as⟩. Note that every coset of K has the form K(as+1 · · · am)t or Kb(as+1 · · · am)t with t ∈ Z p rs+1 s+1 ...p rm m . Since θa1a2...as is the automorphism of D2n induced by a 7→ a and b 7→ ba1a2 . . . as, we have that FD2n(L) = ⟨a⟩, and for every coset of Kc in b⟨a⟩, L is transitive on Kc. Clearly, K ⊴D2n as K is characteristic in ⟨a⟩, and |D2n : FD2n(L)| = 2. 626 Ars Math. Contemp. 23 (2023) #P4.08 / 615–629 Write γ = ε1 . . . εs. Then γ ̸= 1 and γ is the automorphism of D2n induced by aγi = a −1 i for 1 ≤ i ≤ s, a γ j = aj for s+ 1 ≤ j ≤ m, b γ = b. Then γ fixes each coset of K in D2n setwise, and FD2n(⟨γ⟩) ̸= ⟨a⟩ = FD2n(L). Noting that w = R(b)ε1ε2 . . . εsεs+1 . . . εm = R(b)γ ∈ H ≤ A and R(b) ∈ R(D2n) ≤ A, we have γ ∈ A, and hence γ ∈ A1 = Aut(D2n, S). By Proposition 2.4(3), Γ is non-normal, a contradiction. Thus, εk = 1 for all 1 ≤ k ≤ m. By Claim 2, ε = ε1 . . . εm = 1 and n = pr11 p r2 2 · · · prmm | ℓ. It follows that v = R(a)θℓa = R(a) and w = R(b)ε = R(b), im- plying H = ⟨v, w⟩ = ⟨R(a), R(b)⟩ = R(D2n), a contradiction. This completes the proof of the the equivalence of (1) and (3). Now we are ready to finish the proof of Theorem 1.1 by proving that (2) and (3) are equivalent, that is, D2n is an NCI-group if and only if either n = 2, 4 or n is odd. The sufficiency follows from the fact that an NDCI-group is an NCI-group, and the necessity follows from Lemma 4.1. Proof of Corollary 1.2. Let the dihedral group D2n be a DCI-group. Then D2n is an NDCI-group. By Theorem 1.1, n is 2, 4 or odd. By Lemma 4.1, D8 is a non-CI-group and so a non-DCI-group. If n is odd, by [13, Theorem 1.2], n is square-free. Thus, n = 2 or n is odd-square-free. Now let D2n be a CI-group. Then D2n is an NCI-group. Again by Theorem 1.1, either n is 2, 4, or n is odd. By [13, Theorem 1.2], if n is odd, then n = 9 or n is square-free. Since D8 is a non-CI-group, we have n = 2, 9, or n is odd-square-free. ORCID iDs Jin-Hua Xie https://orcid.org/0000-0002-6547-0885 Yan-Quan Feng https://orcid.org/0000-0003-3214-0609 Jin-Xin Zhou https://orcid.org/0000-0002-8353-896X References [1] A. Ádám, Research problem 2-10, J. Comb. Theory 2 (1967), 393, doi: 10.1016/S0021-9800(67)80037-1, https://doi.org/10.1016/S0021-9800(67) 80037-1. [2] B. Alspach and L. A. Nowitz, Elementary proofs that Z2p and Z3p are CI-groups, European J. Comb. 20 (1999), 607–617, doi:10.1006/eujc.1999.0309, https://doi.org/10.1006/ eujc.1999.0309. [3] B. Alspach and T. D. Parsons, Isomorphism of circulant graphs and digraphs, Discrete Math. 25 (1979), 97–108, doi:10.1016/0012-365X(79)90011-6, https://doi.org/10.1016/ 0012-365X(79)90011-6. [4] L. Babai, Isomorphism problem for a class of point-symmetric structures, Acta Math. Acad. Sci. Hungar. 29 (1977), 329–336, doi:10.1007/bf01895854, https://doi.org/10.1007/ bf01895854. [5] S. Bhoumik, E. Dobson and J. Morris, On the automorphism groups of almost all circulant graphs and digraphs, Ars Math. Contemp. 7 (2014), 499–518, doi:10.26493/1855-3974.315. 868, https://doi.org/10.26493/1855-3974.315.868. J.-H. Xie et al.: Normal Cayley digraphs of dihedral groups with the CI-property 627 [6] N. Biggs, Algebraic Graph Theory, Cambridge Mathematical Library, Cambridge Univer- sity Press, Cambridge, 2nd edition, 1993, doi:10.1017/cbo9780511608704, https://doi. org/10.1017/cbo9780511608704. [7] W. Bosma, J. Cannon and C. Playoust, The Magma algebra system. I. The user language, vol- ume 24, pp. 235–265, 1997, doi:10.1006/jsco.1996.0125, computational algebra and number theory (London, 1993), https://doi.org/10.1006/jsco.1996.0125. [8] M. Conder and C. H. Li, On isomorphisms of finite Cayley graphs, Eur. J. Comb. 19 (1998), 911–919, doi:10.1006/eujc.1998.0243, https://doi.org/10.1006/eujc. 1998.0243. [9] J. D. Dixon and B. Mortimer, Permutation Groups, volume 163 of Graduate Texts in Mathe- matics, Springer-Verlag, New York, 1996, doi:10.1007/978-1-4612-0731-3, https://doi. org/10.1007/978-1-4612-0731-3. [10] D. Ź. Djoković, Isomorphism problem for a special class of graphs, Acta Math. Acad. Sci. Hungar. 21 (1970), 267–270, doi:10.1007/bf01894773, https://doi.org/10.1007/ bf01894773. [11] E. Dobson, Isomorphism problem for Cayley graphs of Z3p , Discrete Math. 147 (1995), 87–94, doi:10.1016/0012-365X(95)00099-I, https://doi.org/10.1016/0012-365X(95) 00099-I. [12] E. Dobson, J. Morris and P. Spiga, Further restrictions on the structure of finite DCI-groups: an addendum, J. Algebraic Combin. 42 (2015), 959–969, doi:10.1007/s10801-015-0612-3, https://doi.org/10.1007/s10801-015-0612-3. [13] T. Dobson, M. Muzychuk and P. Spiga, Generalised dihedral CI-groups, Ars Math. Contemp. 22 (2022), Paper No. 7, 18 pp., doi:10.26493/1855-3974.2443.02e, https://doi.org/ 10.26493/1855-3974.2443.02e. [14] J.-L. Du and Y.-Q. Feng, Tetravalent 2-arc-transitive Cayley graphs on non-abelian sim- ple groups, Comm. Algebra 47 (2019), 4565–4574, doi:10.1080/00927872.2018.1549661, https://doi.org/10.1080/00927872.2018.1549661. [15] J.-L. Du, Y.-Q. Feng and J.-X. Zhou, Pentavalent symmetric graphs admitting vertex-transitive non-abelian simple groups, Eur. J. Comb. 63 (2017), 134–145, doi:10.1016/j.ejc.2017.03.007, https://doi.org/10.1016/j.ejc.2017.03.007. [16] B. Elspas and J. Turner, Graphs with circulant adjacency matrices, J. Comb. Theory 9 (1970), 297–307, doi:10.1016/S0021-9800(70)80068-0, https://doi.org/10.1016/ S0021-9800(70)80068-0. [17] X. Fang, X. Ma and J. Wang, On locally primitive Cayley graphs of finite simple groups, J. Comb. Theory Ser. A 118 (2011), 1039–1051, doi:10.1016/j.jcta.2010.10.008, https:// doi.org/10.1016/j.jcta.2010.10.008. [18] X.-G. Fang, Z.-P. Lu, J. Wang and M.-Y. Xu, Cayley digraphs of finite simple groups with small out-valency, Comm. Algebra 32 (2004), 1201–1211, doi:10.1081/agb-120027974, https: //doi.org/10.1081/agb-120027974. [19] Y.-Q. Feng, Automorphism groups of Cayley graphs on symmetric groups with generating transposition sets, J. Comb. Theory Ser. B 96 (2006), 67–72, doi:10.1016/j.jctb.2005.06.010, https://doi.org/10.1016/j.jctb.2005.06.010. [20] Y.-Q. Feng and I. Kovács, Elementary abelian groups of rank 5 are DCI-groups, J. Comb. Theory Ser. A 157 (2018), 162–204, doi:10.1016/j.jcta.2018.02.003, https://doi.org/ 10.1016/j.jcta.2018.02.003. [21] C. D. Godsil, On the full automorphism group of a graph, Combinatorica 1 (1981), 243–256, doi:10.1007/bf02579330, https://doi.org/10.1007/bf02579330. 628 Ars Math. Contemp. 23 (2023) #P4.08 / 615–629 [22] C. D. Godsil, On Cayley graph isomorphisms, Ars Comb. 15 (1983), 231–246. [23] R. Hammack, W. Imrich and S. Klavžar, Handbook of Product Graphs, Discrete Mathematics and its Applications (Boca Raton), CRC Press, Boca Raton, FL, 2nd edition, 2011, doi:10. 1201/b10959, with a foreword by Peter Winkler, https://doi.org/10.1201/b10959. [24] D. Holt and G. Royle, A census of small transitive groups and vertex-transitive graphs, J. Symbolic Comput. 101 (2020), 51–60, doi:10.1016/j.jsc.2019.06.006, https://doi.org/ 10.1016/j.jsc.2019.06.006. [25] S.-S. Jasha, Automorphism groups for semidirect products of cyclic groups, 2019, arXiv:1906.05901v1 [math.CO]. [26] I. Kovács and G. Ryabov, The group C4p × Cq is a DCI-group, Discrete Math. 345 (2022), Pa- per No. 112705, 15, doi:10.1016/j.disc.2021.112705, https://doi.org/10.1016/j. disc.2021.112705. [27] C. H. Li, On isomorphisms of finite Cayley graphs—a survey, Discrete Math. 256 (2002), 301–334, doi:10.1016/S0012-365X(01)00438-1, https://doi.org/10.1016/ S0012-365X(01)00438-1. [28] C. H. Li, Z. P. Lu and P. P. Pálfy, Further restrictions on the structure of finite CI-groups, J. Algebraic Combin. 26 (2007), 161–181, doi:10.1007/s10801-006-0052-1, https://doi. org/10.1007/s10801-006-0052-1. [29] M. Muzychuk, Ádám’s conjecture is true in the square-free case, J. Combin. Theory Ser. A 72 (1995), 118–134, doi:10.1016/0097-3165(95)90031-4, https://doi.org/10.1016/ 0097-3165(95)90031-4. [30] M. Muzychuk, On Ádám’s conjecture for circulant graphs, Discrete Math. 176 (1997), 497–510, doi:10.1016/s0012-365x(96)00251-8, https://doi.org/10.1016/ s0012-365x(96)00251-8. [31] M. Muzychuk, An elementary abelian group of large rank is not a CI-group, Discrete Math. 264 (2003), 167–185, doi:10.1016/s0012-365x(02)00558-7, https://doi.org/10.1016/ s0012-365x(02)00558-7. [32] L. A. Nowitz, A non-Cayley-invariant Cayley graph of the elementary abelian group of or- der 64, Discrete Math. 110 (1992), 223–228, doi:10.1016/0012-365x(92)90711-n, https: //doi.org/10.1016/0012-365x(92)90711-n. [33] P. P. Pálfy, Isomorphism problem for relational structures with a cyclic automorphism, Eur. J. Comb. 8 (1987), 35–43, doi:10.1016/s0195-6698(87)80018-5, https://doi.org/10. 1016/s0195-6698(87)80018-5. [34] J. Pan, F. Yin and B. Ling, Arc-transitive Cayley graphs on non-abelian simple groups with soluble vertex stabilizers and valency seven, Discrete Math. 342 (2019), 689–696, doi:10.1016/ j.disc.2018.11.002, https://doi.org/10.1016/j.disc.2018.11.002. [35] D. J. S. Robinson, A Course in the Theory of Groups, volume 80 of Graduate Texts in Mathematics, Springer-Verlag, New York, 2nd edition, 1996, doi:10.1007/978-1-4419-8594-1, https://doi.org/10.1007/978-1-4419-8594-1. [36] J. J. Rotman, An Introduction to the Theory of Groups, volume 148 of Graduate Texts in Mathematics, Springer-Verlag, New York, 4th edition, 1995, doi:10.1007/978-1-4612-4176-8, https://doi.org/10.1007/978-1-4612-4176-8. [37] G. Somlai, Elementary abelian p-groups of rank 2p + 3 are not CI-groups, J. Algebraic Comb. 34 (2011), 323–335, doi:10.1007/s10801-011-0273-9, https://doi.org/10. 1007/s10801-011-0273-9. J.-H. Xie et al.: Normal Cayley digraphs of dihedral groups with the CI-property 629 [38] G. Somlai and M. Muzychuk, The Cayley isomorphism property for Z3p × Zq , Algebr. Comb. 4 (2021), 289–299, doi:10.5802/alco.154, https://doi.org/10.5802/alco.154. [39] P. Spiga, Elementary abelian p-groups of rank greater than or equal to 4p − 2 are not CI- groups, J. Algebraic Comb. 26 (2007), 343–355, doi:10.1007/s10801-007-0059-2, https: //doi.org/10.1007/s10801-007-0059-2. [40] P. Spiga, CI-property of elementary abelian 3-groups, Discrete Math. 309 (2009), 3393– 3398, doi:10.1016/j.disc.2008.08.002, https://doi.org/10.1016/j.disc.2008. 08.002. [41] J. Turner, Point-symmetric graphs with a prime number of points, J. Comb. Theory 3 (1967), 136–145, doi:10.1016/S0021-9800(67)80003-6, https://doi.org/10.1016/ S0021-9800(67)80003-6. [42] J.-H. Xie, Y.-Q. Feng and Y. S. Kwon, Normal Cayley digraphs of generalized quaternion groups with CI-property, Appl. Math. Comput. 422 (2022), Paper No. 126966, 9, doi:10.1016/ j.amc.2022.126966, https://doi.org/10.1016/j.amc.2022.126966. [43] J.-H. Xie, Y.-Q. Feng, G. Ryabov and Y.-L. Liu, Normal Cayley digraphs of cyclic groups with CI-property, Comm. Algebra 50 (2022), 2911–2920, doi:10.1080/00927872.2021.2022156, https://doi.org/10.1080/00927872.2021.2022156. [44] M.-Y. Xu, Automorphism groups and isomorphisms of Cayley digraphs, Discrete Math. 182 (1998), 309–319, doi:10.1016/s0012-365x(97)00152-0, https://doi.org/10.1016/ s0012-365x(97)00152-0. [45] F.-G. Yin, Y.-Q. Feng, J.-X. Zhou and S.-S. Chen, Arc-transitive Cayley graphs on nonabelian simple groups with prime valency, J. Comb. Theory Ser. A 177 (2021), Paper No. 105303, 20 pp., doi:10.1016/j.jcta.2020.105303, https://doi.org/10.1016/j.jcta.2020. 105303. [46] J.-X. Zhou, The automorphism group of the alternating group graph, Appl. Math. Lett. 24 (2011), 229–231, doi:10.1016/j.aml.2010.09.009, https://doi.org/10.1016/j. aml.2010.09.009. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.09 / 631–648 https://doi.org/10.26493/1855-3974.2815.1e7 (Also available at http://amc-journal.eu) On cubic bi-Cayley graphs of p-groups* Na Li Department of Mathematics, Beijing Jiaotong University, Beijing, 100044, China Young Soo Kwon † Department of Mathematics, Yeungnam University, 280 Daehak-ro, Gyeongsan, Gyeongbuk 38541, Korea Jin-Xin Zhou ‡ Department of Mathematics, Beijing Jiaotong University, Beijing, 100044, China Received 26 January 2022, accepted 27 April 2023, published online 21 June 2023 Abstract A graph is called a Cayley graph (or bi-Cayley graph, respectively) of a group G if it has a group G of automorphisms acting semiregularly on the vertices with exactly one orbit (or two orbits, respectively). It is known every Cayley graph is vertex-transitive. In this paper, we first present a classification of connected cubic non-Cayley vertex-transitive bi-Cayley graphs of a finite p-group H , where p > 3 is a prime and the derived subgroup of H is either cyclic or isomorphic to Zp × Zp. This is then used to give a classification of connected cubic non-Cayley vertex-transitive graphs of order 2p4 for each prime p. Keywords: Bi-Cayley graph, vertex-transitive, non-Cayley. Math. Subj. Class. (2020): 05C25, 20B25 *The authors are grateful to the referees for helpful remarks and suggestions. This work was partially supported by the Fundamental Research Funds for the Central Universities (2022JBCG003), the National Natural Science Foundation of China (12071023, 12161141005). †The second author was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2018R1D1A1B05048450). ‡Corresponding author. E-mail addresses: 20118007@bjtu.edu.cn (Na Li), ysookwon@ynu.ac.kr (Young Soo Kwon), jxzhou@bjtu.edu.cn (Jin-Xin Zhou) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 632 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 1 Introduction In this paper, we shall describe an investigation of non-Cayley vertex-transitive bi-Cayley graphs of finite p-groups. To explain this, we first introduce some terminology. Let G be a permutation group on a set Ω and α ∈ Ω. The stabilizer of α in G is the subgroup Gα = {g ∈ G | αg = α}. If Gα = 1 for every α ∈ Ω, then G is said to be semiregular on Ω, and if G is transitive and semiregular on Ω, then we say that G is regular on Ω. Let Γ be a finite simple connected graph with vertex set V (Γ) and E(Γ). We use Aut(Γ) to denote the full automorphism group of Γ. Note that Aut(Γ) is a permutation group on V (Γ). A graph Γ is vertex-transitive if Aut(Γ) is transitive on V (Γ), and a vertex-transitive graph Γ is called non-Cayley vertex-transitive if Aut(Γ) has no regular subgroups. Let R,L and S be subsets of a group H such that R = R−1, L = L−1 and R ∪ L does not contain the identity element of H . The bi-Cayley graph BiCay(H,R,L, S) of H relative to R,L, S is a graph having vertex set the union of the following two copies of H: H0 = {h0 | h ∈ H} and H1 = {h1 | h ∈ H}, and edge set {{h0, g0} | gh−1 ∈ R} ∪ {{h1, g1} | gh−1 ∈ L} ∪ {{h0, g1} | gh−1 ∈ S}. In the study of bi-Cayley graphs, much work has been devoted to construct and clas- sify non-Cayley vertex-transitive graphs, see, for example, [19, 20, 23, 25]. In [25], the following problem was proposed: Problem A ([25, Problem 1]). Characterize cubic non-Cayley vertex-transitive bi-Cayley graphs of a p-group for an odd prime p. In [25], Problem A was partially solved for cubic bi-Cayley graphs of regular p-groups. In this paper, we make a progress towards Problem A by classifying non-Cayley vertex- transitive bi-Cayley graphs of a p-group H such that the derived group of H is cyclic or isomorphic to Zp × Zp, where p > 3 is prime. The following is our first main result. Theorem 1.1. Let p > 3 be a prime and let H be a finite p-group such that the derived subgroup of H is either cyclic or isomorphic to Zp × Zp. Let Γ be a connected cubic bi- Cayley graph of H . Then Γ is non-Cayley vertex-transitive if and only if Γ is isomorphic to one of the graphs given in Constructions I – V (see Section 4). Note that every cubic non-Cayley vertex-transitive graph of order 2p or 2p2 (p a prime) is a generalized Petersen graph, see [13, 21]. In [25], all connected cubic non-Cayley vertex-transitive graphs of order 2p3 were classified for each prime p. Applying the above theorem, our next main theorem gives a classification of connected cubic non-Cayley vertex-transitive graphs of order 2p4 for each prime p. Theorem 1.2. Let p be a prime. Then a connected cubic graph Γ of order 2p4 is a non- Cayley vertex-transitive graph if and only if Γ is isomorphic to Γ(p,m, n, s; t)(m+n+3 = 4,m > n ≥ 0, s ≥ 0), Γ(p, 1; k, i) or ∆(p, 1; k) (see Section 4 for the construction of these graphs). N. Li et al.: On cubic bi-Cayley graphs of p-groups 633 2 Preliminaries We first introduce some notation about groups. For a positive integer, let Zn be the cyclic group of order n and Z∗n be the multiplicative group of Zn consisting of numbers coprime to n. A semidirect product of a group N by a group M is denoted by N ⋊ M . If H is a subgroup of a group G, the centralizer and normalizer of H in G are denoted by CG(H) and NG(H), respectively. The automorphism group, the center, the derived subgroup and the Frattini subgroup of a group G will be represented by Aut(G), Z(G), G′ and Φ(G), respectively. 2.1 Cayley graphs Given a finite group G and a self-inverse subset S ⊆ G \ {1}, the Cayley graph Γ = Cay(G,S) on G relative to S is a graph with vertex set G and edge set {{g, sg} | g ∈ G, s ∈ S}. Let Aut(G,S) := {α ∈ Aut(G)|Sα = S}. Then Aut(G,S) ≤ Aut(Γ)1. For any g ∈ G, R(g) is the permutation of G defined by R(g) : x 7→ xg for x ∈ G. Set R(G) := {R(g) | g ∈ G}. It is well-known that R(G) is a regular subgroup of Aut(Cay(G,S)). By [9, Lemma 2.1], we have NAut(Γ) = R(G) ⋊ Aut(G,S). Clearly, every Cayley graph is vertex-transitive. In general, we have the following proposition. Proposition 2.1 ([1, Lemma 16.3]). A graph Γ is isomorphic to a Cayley graph of a group G if and only if Aut(Γ) has a subgroup isomorphic to G, acting regularly on the vertex set of Γ. The next two propositions about Cayley graphs will be used in the sequel. Proposition 2.2 ([22, Theorem 1.1]). Let G be a finite p-group with an odd prime p, and let Γ = Cay(G,S) be connected and of valency 4. Assume that Γ is X-edge-transitive, where G ≤ X ≤ Aut(Γ). Then either G is normal in X , or X has a normal subgroup R such that R ≤ G, and Γ is a normal cover of ΓR, and moreover, the quotient graph ΓR and the quotient group X̄ satisfy one of the following conditions (1) and (2) (Here, for any R ≤ Y ≤ X and x ∈ X , let Ȳ = Y/R and x̄ = xR.): (1) ΓR ∼= K5(the complete graph of order 5) and A5 ≤ X̄ ≤ S5. (2) X̄ has a unique minimal normal subgroup N̄ ∼= Znp with n = pm such that (i) Ḡ = N̄ ⋊ M̄ , where M̄ ∼= Zpm with m ⩾ 1, (ii) X̄ = N̄ ⋊ ((H̄⋊M̄).Ō) and X̄1Ḡ = H̄.Ō, where H̄ ∼= Z l 2 with 2 ≤ l ≤ n and Ō ∼= Zt with t = 1 or 2, such that there exist b̄1, ..., b̄n ∈ N̄ and h̄1, ..., h̄n ∈ H̄ such that N̄ = ⟨b̄1, ..., b̄n⟩, ⟨b̄i, h̄i⟩ ∼= D2p and H̄ = ⟨h̄i⟩ × CH̄(b̄i) for 1 ≤ i ≤ n. A graph Γ is called symmetric if Aut(Γ) is transitive on the arcs of Γ. By [8, Corol- lary 3.4], we have the following result. Proposition 2.3. Every connected cubic symmetric graph of order 2pn is a Cayley graph whenever n ≥ 1 and p ≥ 7 is a prime. 634 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 2.2 Basic properties of bi-Cayley graphs In this subsection, we let Γ be a connected bi-Cayley graph BiCay(H,R,L, S) of a group H . The following lemma provides some basic properties of Γ (see [24, Lemma 3.1]). Lemma 2.4. The following hold. (1) H is generated by R ∪ L ∪ S. (2) Up to graph isomorphism, S can be chosen to contain the identity of H . For each g ∈ H , let R(g) : hi 7→ (hg)i, ∀i ∈ Z2, h ∈ H. (2.1) Set R(H) = {R(g) | g ∈ H}. Then R(H) is a semiregular subgroup of Aut(Γ) with H0 and H1 as its two orbits. In general, a graph Γ is isomorphic to a bi-Cayley graph of a group H if and only if Γ admits a group of automorphisms which is isomorphic to H and acts semiregularly on the vertices with two orbits. For α ∈ Aut(H) and x, y, g ∈ H , let δα,x,y : h0 7→ (xhα)1, h1 7→ (yhα)0, ∀h ∈ H, σα,g : h0 7→ (hα)0, h1 7→ (ghα)1, ∀h ∈ H. (2.2) Set I = {δα,x,y | α ∈ Aut(H) s.t. Rα = x−1Lx, Lα = y−1Ry, Sα = y−1S−1x}, F = {σα,g | α ∈ Aut(H) s.t. Rα = R, Lα = g−1Lg, Sα = g−1S}. (2.3) The following theorem determines the normalizer of R(H) in Aut(Γ). Theorem 2.5 ([24, Theorem 1.1]). Let Γ = BiCay(H,R,L, S) be a connected bi-Cayley graph of a group H . Then NAut(Γ)(R(H)) = R(H)⋊F if I = ∅ and NAut(Γ)(R(H)) = R(H)⟨F, δα,x,y⟩ if I ̸= ∅ and δα,x,y ∈ I . Furthermore, if I ̸= ∅, then for any δα,x,y ∈ I , we have (1) ⟨R(H), δα,x,y⟩ acts transitively on V (Γ); (2) if α has order 2 and x = y = 1, then Γ is isomorphic to the Cayley graph Cay(H̄, R ∪ αS), where H̄ = H ⋊ ⟨α⟩. 2.3 Normal covers Let Γ be a graph. Assume that G ≤ Aut(Γ) is vertex-transitive on Γ. Let N be a normal subgroup of G such that N is intransitive on V (Γ). The normal quotient graph ΓN of Γ relative to N is defined as the graph with vertices the orbits of N on V (Γ) and with two different orbits adjacent if there exists an edge in Γ between the vertices lying in those two orbits. If ΓN and Γ have the same valency, then we say that Γ is a normal N -cover of ΓN . By [12, Lemma 2.5], we have the following lemma. Lemma 2.6. Let Γ be a connected cubic graph such that G ≤ Aut(Γ) is arc-transitive on Γ. Suppose that there exists N G such that N has at least three orbits on V (Γ). Then N. Li et al.: On cubic bi-Cayley graphs of p-groups 635 (1) Γ is a normal N -cover of the quotient graph ΓN of Γ relative to N . (2) N acts semiregularly on V (Γ), N is the kernel of G acting on V (ΓN ) and G/N ≤ Aut(ΓN ). (3) G/N is arc-transitive on ΓN . 3 Cubic bi-Cayley graphs of groups of odd order In this section, we shall give a description of cubic bi-Cayley graphs of groups of odd order, and prove a sufficient condition for such graphs being non-Cayley vertex-transitive. Lemma 3.1. Let Γ be a connected cubic non-Cayley vertex-transitive graph. Let A ≤ Aut(Γ) be vertex- but not arc-transitive on Γ. Take any vertex v of Γ. Then there exists a unique neighbor v∗ such that Av = Av∗ . In particular, {v, v∗} is a block of imprimitivity of A on V (Γ). Proof. Since Γ is not a Cayley graph, one has Av > 1. Assume that |Av| is divisible by p, where p > 3 is an odd prime. Then Av contains an element α of order p. Note that each orbit of ⟨α⟩ has length either 1 or p. Since ⟨α⟩ fixes v and Γ has valency 3, the con- nectedness of Γ implies that each orbit of ⟨α⟩ has length 1, and so α = 1, a contradiction. Since Γ is not arc-transitive, it follows that 3 ∤ |Av|. Thus Av is a nontrivial 2-group for every v ∈ V (Γ). Let A∗v be the kernel of Av acting on the neighborhood of v in Γ. Then Av/A ∗ v ≤ Z2. If Av/A∗v = 1, then Av = A∗v fixes all neighbors of v, and then since Γ is connected and A is vertex-transitive on Γ, Av would fix all vertices of Γ, forcing Av = 1, a contradiction. Thus, Av/A∗v ∼= Z2. So there exists a unique neighbor, say v∗, of v such that Av = Av∗ . For any g ∈ A, it is easily verified that Avg = Agv = A g v∗ = A(v∗)g . It follows that either {v, v∗} = {vg, (v∗)g} or {v, v∗} ∩ {vg, (v∗)g} = ∅. So {v, v∗} is a block of imprimitivity of A on V (Γ). Definition 3.2. Let Γ be a connected cubic non-Cayley vertex-transitive graph. Let A ≤ Aut(Γ) be vertex- but not arc-transitive on Γ. Set M(Γ) = {{v, v∗} | v, v∗ ∈ V (Γ), v ∼ v∗, Av = Av∗}. The matching quotient graph ΓM of Γ is the graph with vertex set M(Γ) and with two elements in M(Γ) adjacent in ΓM if there exists an edge of Γ between them. In the above definition, it is easy to see that M(Γ) is A-invariant and A induces a group of automorphisms of ΓM. By [14, Lemma 9 & Theorem 10], we have the following lemma. Lemma 3.3. Let Γ be a connected cubic non-Cayley vertex-transitive graph, and let A ≤ Aut(Γ) be vertex- but not arc-transitive on Γ. Then ΓM is a connected tetravalent A-arc- transitive graph, that is, A acts faithfully on M(Γ) and is arc-transitive ΓM. The following proposition gives a description of connected cubic bi-Cayley graphs of groups of odd order. 636 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 Proposition 3.4. Let Γ = BiCay(H,R,L, S) be a connected cubic bi-Cayley graph of a group H of odd order. Let A ≤ Aut(Γ) be such that R(H) ≤ A. If Γ is a non- Cayley vertex-transitive graph and A is vertex- but not arc-transitive on Γ, then one of the following holds. (1) Γ ∼= BiCay(H, ∅, ∅, {1, x, y}), and ΓM ∼= Cay(H, {x, y, x−1, y−1}), where H = ⟨x, y⟩. Furthermore, H has no automorphism α such that xα = x−1 and yα = y−1. In particular, R(H) is not normal in A. (2) Γ ∼= BiCay(H, {a, a−1}, {b, b−1}, {1}), and ΓM ∼= Cay(H, {a, b, a−1, b−1}), where H = ⟨a, b⟩. Furthermore, H has no automorphism α such that aα = a−1 and bα = b−1. Proof. Assume that Γ is a non-Cayley vertex-transitive graph. Clearly, Γ has 2|H| vertices as Γ is a bi-Cayley of the group H . Since A ≤ Aut(Γ) is vertex- but not arc-transitive on Γ, by Lemma 3.3, A acts faithfully on M(Γ) (see Definition 3.2). As H has odd order, for any {u, u∗} ∈ M(Γ), we have |{u, u∗} ∩Hi| = 1 with i = 1, 2, where Hi(i = 0, 1) are the two orbits of R(H) on V (Γ). It follows that R(H) acts regularly on M(Γ), and so ΓM is a Cayley graph of H . By Lemma 2.4, we may assume that S contains the identity 1 of H . Then {10, 11} is an edge of Γ. Since H has odd order, we have |R| = |L| = 0 or 2. If |R| = |L| = 0, then we may let S = {1, x, y}. Since Γ is connected, by Lemma 2.4 we have H = ⟨S⟩ = ⟨x, y⟩. We may assume that {10, s1} ∈ M(Γ) with s ∈ S. Then A10 = As1 . Define a map on V (Γ) as follows: L(s) : h0 7→ h0, h1 7→ (s−1h)1,∀h ∈ H. It is easy to verify that L(s) is a permutation on V (Γ). Let Γ′ = BiCay(H, ∅, ∅, s−1S). For each edge {h0, g1} of Γ, we have gh−1 ∈ S and so s−1gh−1 ∈ s−1S. It fol- lows that {h0, (s−1g)1} is an edge of Γ′. Clearly, both Γ and Γ′ have valency 3, so L(s) is an isomorphism between Γ and Γ′. For any edge e of Γ′ and for any α ∈ A, we have eL(s) −1αL(s) ∈ E(Γ′). This implies that L(s)−1AL(s) ≤ Aut(Γ′). Let B = L(s)−1AL(s). Since L(s) fixes 10, one has B10 = L(s) −1A10L(s). It is easy to see that L(s)−1As1L(s) = B11 . Since A10 = As1 , one has B10 = B11 . Then {11, 10} ∈ M(Γ′). Therefore, without loss of generality, we may assume that {11, 10} ∈ M(Γ). Then we have M(Γ) = {{h0, h1} | h ∈ H}. By Lemma 3.3, ΓM has valency 4. So {x0, x1}, {y0, y1}, {(x−1)0, (x−1)1}, {(y−1)0, (y−1)1} are just the four neighbors of {11, 10} in ΓM. Clearly, {11, 10}R(h) = {h0, h1} for each h ∈ H , so we have ΓM ∼= Cay(H, {x, y, x−1, y−1}). If H has an automorphism, say α, such that xα = x−1 and yα = y−1, then α has order 2 and Sα = S−1, and then by Theorem 2.5, R(H)⋊ ⟨δα,1,1⟩ acts regularly on V (Γ). This is contrary to the fact that Γ is a non-Cayley vertex-transitive graph. By Lemma 3.3, ΓM is A-arc-transitive. If R(H) is normal in A, then Aut(H, {x, y, x−1, y−1}) is transitive on {x, y, x−1, y−1}. So Aut(H) must have an involution inter- changing the two pairs (x, y) and (x−1, y−1), which is impossible by the argument in the above paragraph. Thus, we obtain part (1). Now assume that |R| = |L| = 2. In this case, we must have M(Γ) = {{h0, h1} | h ∈ H}. Assume that R = {a, a−1} and L = {b, b−1}. Again, by Lemma 3.3, ΓM N. Li et al.: On cubic bi-Cayley graphs of p-groups 637 has valency 4. So {a0, a1}, {b0, b1}, {(a−1)0, (a−1)1}, {(b−1)0, (b−1)1} are just the four neighbors of {11, 10} in ΓM. As {11, 10}R(h) = {h0, h1} for each h ∈ H , we have ΓM ∼= Cay(H, {a, b, a−1, b−1}). If H has an automorphism, say α, such that aα = a−1 and bα = b−1, then α has order 2 and Rα = L, and then by Theorem 2.5, R(H)⋊ ⟨δα,1,1⟩ acts regularly on V (Γ). This is contrary to the fact that Γ is a non-Cayley vertex-transitive graph. Then part (2) happens. We now give a sufficient condition for a cubic bi-Cayley graph of a group of odd order in (2) of Proposition 3.4 being non-Cayley vertex-transitive. Theorem 3.5. Let Γ = BiCay(H, {a, a−1}, {b, b−1}, {1}) be a connected cubic bi-Cayley of a group H of odd order, where a, b ∈ H . If Aut(H, {a, b, a−1, b−1}) ∼= Z4, then Γ is a non-Cayley vertex-transitive graph. Proof. Assume that Aut(H, {a, b, a−1, b−1}) ∼= Z4. Then H has an automorphism α such that aα = b and bα = a−1. So α swaps {a, a−1} and {b, b−1}. By Theorem 2.5, ⟨R(H), δα,1,1⟩ acts transitively on V (Γ). Suppose on the contrary that Γ is a Cayley graph. First, assume that Γ is symmetric. Let N be the largest normal subgroup of Aut(Γ) contained in R(H). Note that each orbit Hi(i = 1 or 2) of R(H) on V (Γ) induces a subgraph which is a union of cycles of odd length. If N = R(H), then R(H) ⊴ Aut(Γ) and then each Hi will be a block of imprimitivity of Aut(Γ) on V (Γ). Since Γ is symmetric and connected, it follows that H1 = H2 = V (Γ), a contradiction. This implies that N < R(H) and Γ is non-bipartite. By Lemma 2.6, the quotient graph ΓN of Γ relative to N is a cubic graph with Aut(Γ)/N as an arc-transitive group of automorphisms. Clearly, R(H)/N ≤ Aut(Γ)/N acts semiregularly on V (ΓN ) with two orbits. So ΓN is a bi-Cayley graph of R(H)/N . If R(H)/N has a subgroup, say M/N , which is normal in Aut(ΓN ), then M/NAut(Γ)/N . Since N is the largest normal subgroup of Aut(Γ) contained in R(H), it follows that M/N is trivial. Thus, the only proper subgroup of R(H)/N which is normal in Aut(ΓN ) is the identity subgroup. As ΓN has order 2|H|/|N | and |H| is odd, by [7, Theorem 7.1], either ΓN is 3-arc-regular and has order 6, 10, 110 or 506, or ΓN is 4-arc-regular and has order 14, 506 or 2162. (One may the definition of s-arc-regular graphs in [7, page 145].) Furthermore, since Γ is not bipartite, by inspecting the list of cubic symmetric graphs of order at most 10000 [6], we get that either ΓN ∼= F10 or F506A, or ΓN ∼= F2162A. However, by Magma [3], each of F10,F506A,F2162A is a non-Cayley graph, a contradiction. (For an integer n, if there is a unique cubic symmetric graph of order n up to graph isomorphism, then we use Fn to denote this graph, and if there are more than one cubic symmetric graph of order n up to graph isomorphism, then we use FnA, FnB, etc. to denote the corresponding graphs (see [4]).) Now assume that Γ is nonsymmetric. Then Aut(Γ)10 is a 2-group. Since |Aut(Γ)| = 2|H||Aut(Γ)10 |, R(H) is 2′-Hall subgroup of Aut(Γ). Since Γ is supposed to be a Cayley graph, Aut(Γ) has a subgroup, say G, acting regularly on V (Γ). Then |G| = 2|H|, and since |H| is odd, G is solvable, and so G has a 2′-Hall subgroup, say B. By [10, Theorem A], R(H) and B are conjugate in Aut(Γ). Without loss of generality, we may assume R(H) ≤ G. Then R(H)  G. Then {H0, H1} is G-invariant. As 11 is the unique neighbor of 10 contained in H1, for each b ∈ G, either {10, 11}b = {10, 11} or {10, 11}b ∩ {10, 11} = ∅. Since G is regular on V (Γ), there exists b ∈ G mapping 10 to 11, and then {10, 11}b = {10, 11}. It follows that b interchanges 10 and 11, and hence b is an involution. 638 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 By Theorem 2.5, there exists an automorphism, say α, of H and x, y ∈ H such that δα,x,y swaps 10 and 11. By the definition of δα,x,y (see Eqs. 2.2–2.3), we have x = y = 1 and α swaps {a, a−1} and {b, b−1}. However, since Aut(H, {a, b, a−1, b−1}) ∼= Z4, the only involution in Aut(H, {a, b, a−1, b−1}) must fix both {a, a−1} and {b, b−1} setwise, a contradiction. 4 Five families of cubic VNC-graphs In this section, we shall apply Theorem 3.5 to construct five families of cubic VNC graphs. Construction I Let p be an odd prime, and m > n ≥ 0 and let t be an integer such that t2 ≡ −1(mod pm−n). Let H(p,m, n, s; t) = ⟨x, y, z | xp m = yp m = zp s = 1, z = [x, y], [x, z] =[y, z] = 1, xp n = yp n ⟩, and Γ(p,m, n, s; t) = BiCay(H(p,m, n, s; t), {x, x−1}, {yt, y−t}, {1}). Lemma 4.1. The graph Γ(p,m, n, s; t) is a connected cubic non-Cayley vertex-transitive graph with 2pm+n+s vertices. Proof. Let G = H(p,m, n, s; t) and Σ = Γ(p,m, n, s; t). Clearly, G′ = ⟨z⟩ ∼= Zps and G/G′ = ⟨xG′, yG′⟩. By the definition of H(p,m, n, s; t), we have ⟨x⟩∩⟨z⟩ = ⟨y⟩∩⟨z⟩ = 1. Since xp n = yp n , one has G/G′ = ⟨xG′⟩ × ⟨xy−1G′⟩ ∼= Zpm × Zpn . So G has order pm+n+s, and so Σ has 2pm+n+s vertices. Let u = yt, v = x−r and w = [u, v], where r ∈ Z∗pm is such that rt ≡ 1(mod pm). Note that t2 ≡ −1(mod pm−n). Clearly, upm = vpm = 1. Since z = [x, y] ∈ Z(G), one has w = [u, v] = [yt, x−r] = [y, x]−tr = ztr, and so wp s = 1. Also, [u,w] = [v, w] = 1. Since rt ≡ 1(mod pm) and t2 ≡ −1(mod pm−n), one has r = −t(mod pm−n). So upn = ytp n = xtp n = x−rp n = vp n . Thus, u, v, w have the same relations as do x, y, z. So G has an automorphism, say α, such that xα = yt and yα = x−r. Then (yt)α = x−tr = x−1, and then α ∈ Aut(G, {x, x−1, yt, y−t}). To complete the proof, by Proposition 3.4, it suffices to prove that Aut(G, {x, x−1, yt, y−t}) = ⟨α⟩. By way of contradiction, assume that Aut(G, {x, x−1, yt, y−t}) > ⟨α⟩. Then there would exist β ∈ Aut(G, {x, x−1, yt, y−t}) such that xβ = yt and (yt)β = x. Then (xH ′)β = ytH ′ and (ytH ′)β = xH ′. Since yH ′ = x · (x−1y)H ′, one has (xH ′)β = ytH ′ = xt · (x−1y)tH ′. It follows that xH ′ = (ytH ′)β = (xtH ′)β · ((x−1y)tH ′)β = xt 2 H ′ · (x−1y)t 2 H ′ · ((x−1y)t)βH ′. As xp n = yp n , one has (x−1y)p n H ′ = H ′, and hence xp n H ′ = xt 2pnH ′. It follows that x(t 2−1)pn ∈ H ′, and since ⟨x⟩ ∩ H ′ = 1, one has x(t2−1)pn = 1. This implies that t2 ≡ 1(mod pm−n). However, it is assumed that t2 ≡ −1(mod pm−n) and m > n. This forces that pm−n | 2, a contradiction. Construction II Let p be an odd prime, and m be an integer. Take i, k ∈ Z∗p and assume that k2 ≡ −1(mod p). Let H(p,m; k, i) = ⟨a, b, c | ap m+1 = cp = 1, ap m = bkp m , c = [a, b], [a, c] =aip m , [b, c] = bip m ⟩ N. Li et al.: On cubic bi-Cayley graphs of p-groups 639 and let Γ(p,m; k, i) = BiCay(H(p,m; k, i), {a, a−1}, {b, b−1}, {1}). Lemma 4.2. The graph Γ(p,m; k, i) is a connected cubic non-Cayley vertex-transitive graph with 2p2m+2 vertices. Proof. Let G = H(p,m; k, i) and let Σ = G(p,m; k, i). Clearly, G′ = ⟨c⟩ × ⟨apm⟩ ∼= Zp × Zp and ap m ∈ Z(G). Letting N = ⟨apm⟩, we have G/N = ⟨aN, bN, cN | ap m N = bp m N = cpN = N, cN = [a, b]N,[cN, aN ] = [cN, bN ] = N⟩. So |G/N | = p2m+1 and so |G| = p2m+2. Thus, Σ has 2p2m+2 vertices. Let u = b, v = a−1 and w = [u, v]. Since ap m = bkp m and k2 ≡ −1(mod p), one has bp m = a−kp m . So we have up m = vkp m and up m+1 = 1. As w = [u, v] = [b, a−1] = [a, b]a −1 = ca −1 , one has wp = 1. As ap m = bkp m , one has [a, c] = aip m ∈ Z(G) and [b, c] = bip m ∈ Z(G). It follows that [u,w] = [b, ca−1 ] = [b, [a, c]c] = [b, c] = bipm = uip m , and [v, w] = [a−1, ca −1 ] = [a−1, [a, c]c] = [a−1, c] = [a, c]−1 = a−ip m = vip m . Thus u, v, w have the same relations as do a, b, c. So H has an automorphism, say α, such that aα = b and bα = a−1. This implies α ∈ Aut(G, {a, a−1, b, b−1}). To complete the proof, by Proposition 3.4, it suffices to prove that Aut(G, {a, a−1, b, b−1}) = ⟨α⟩. Suppose on the contrary that Aut(G, {a, a−1, b, b−1}) > ⟨α⟩. Then there would exist β ∈ Aut(G, {a, a−1, b, b−1}) such that aβ = b and bβ = a. Then bpm = (apm)β = (bkp m )β = akp m . It follows that ap m = ak 2pm and hence k2 ≡ 1(mod p). However, since it is assumed that k2 ≡ −1(mod p), one has p | 2, a contradiction. Construction III Let p be an odd prime, and m > n > 0 be integers. Take i ∈ Z∗p and s ∈ Z∗pn such that s2 ≡ −1(mod pn). Let H(p,m, n; s, i) be a group with the follow presentation: ⟨a, b, c | ap m+1 = cp = 1, ap m−n = bsp m−n , c = [a, b], [a, c] = aip m , [b, c] = bip m ⟩ and Γ(p,m, n; s, i) = BiCay(H(p,m, n; s, i), {a, a−1}, {b, b−1}, {1}). Lemma 4.3. The graph Γ(p,m, n; s, i) is a connected cubic non-Cayley vertex-transitive graph with 2p2m−n+2 vertices. Proof. Let G = H(p,m, n; s, i) and let Σ = Γ(p,m, n; s, i). Clearly, G′ = ⟨c⟩×⟨apm⟩ ∼= Zp ×Zp, ap m ∈ Z(G) and G/G′ = ⟨aG′, bG′⟩. Since apm , bpm ∈ G′, one has G′ ∩ ⟨a⟩ = G′ ∩ ⟨b⟩ = ⟨apm⟩. Since apm−n = bspm−n , one has ⟨aG′⟩ ∩ ⟨bG′⟩ = ⟨apm−nG′⟩ ∼= Zpn , and so |G/G′| = p2m−n. It follows that G/G′ = ⟨aG′⟩× ⟨ab−sG′⟩ ∼= Zpm ×Zpm−n , and hence |G| = p2m−n+2. Thus, Σ has 2p2m−n+2 vertices. Let u = b, v = a−1 and w = [u, v]. Since ap m−n = bsp m−n and s2 ≡ −1(mod pn), one has bp m−n = a−sp m−n . So we have up m−n = vsp m−n and up m+1 = 1. As w = [u, v] = [b, a−1] = [a, b]a −1 = ca −1 , one has wp = 1. As ap m−n = bsp m−n , one has [a, c] = aip m ∈ Z(G) and [b, c] = bipm ∈ Z(G). It follows that [u,w] = [b, ca−1 ] = [b, [a, c]c] = [b, c] = bip m = uip m , and [v, w] = [a−1, ca −1 ] = [a−1, [a, c]c] = [a−1, c] = [a, c]−1 = a−ip m = vip m . Thus u, v, w have the same relations as do a, b, c. So G has an automorphism, say α, such that aα = b and bα = a−1, and hence α ∈ Aut(G, {a, a−1, b, 640 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 b−1}). To complete the proof, by Proposition 3.4, it suffices to prove that Aut(G, {a, a−1, b, b−1}) = ⟨α⟩. Suppose on the contrary that Aut(G, {a, a−1, b, b−1}) > ⟨α⟩. Then there would exist β ∈ Aut(G, {a, a−1, b, b−1}) such that aβ = b and bβ = a. Then bpm−n = (apm−n)β = (bsp m−n )β = asp m−n . It follows that ap m−n = as 2pm−n and hence s2 ≡ 1(mod pn). However, since it is assumed that s2 ≡ −1(mod pn), one has p | 2, a contradiction. Construction IV Let p be an odd prime, and m be an integer. Take k ∈ Z∗p such that k2 ≡ −1(mod p). Let H(p,m; k) be a group with the following representation: ⟨a, b, c, d | ap m = bp m = cp = dp = 1, c = [a, b], [a, c] = d, [b, c] = dk, [a, d] = [b, d] = 1⟩ and let ∆(p,m; k) = BiCay(H(p,m; k), {a, a−1}, {b, b−1}, {1}). Lemma 4.4. The graph ∆(p,m; k) is a connected cubic non-Cayley vertex-transitive graph with 2p2m+2. Proof. Let G = H(p,m; k) and let Σ = ∆(p,m; k). Clearly, G′ = ⟨c⟩ × ⟨d⟩ ∼= Zp × Zp, d ∈ Z(G) and G/G′ = ⟨aG′, bG′⟩. Letting N = ⟨d⟩, we have G/N = ⟨aN, bN, cN | ap m N = bp m N = cpN = N, cN = [a, b]N,[cN, aN ] = [cN, bN ] = N⟩. So |G/N | = p2m+1 and so |G| = p2m+2. Thus, Σ has 2p2m+2 vertices. Let u = b, v = a−1, x = [u, v] and y = [u, x]. Clearly, up m = vp m = 1. Note that x = [u, v] = [b, a−1] = [a, b]a −1 = aca−1c−1ac(ac)−1c = d(ac) −1 c = dc. Then y = [u, x] = [b, dc] = [b, c] = dk. It follows that xp = yp = 1. Furthermore, [v, x] = [a−1, c] = [c, a]a −1 = d−1 = dk 2 = yk. Clearly, [u, y] = [v, y] = 1 due to [a, d] = [b, d] = 1. Thus u, v, x, y have the same relations as do a, b, c, d. So G has an automorphism, say α, such that aα = b and bα = a−1. This implies α ∈ Aut(G, {a, a−1, b, b−1}). To complete the proof, by Proposition 3.4, it suffices to prove that Aut(G, {a, a−1, b, b−1}) = ⟨α⟩. Suppose on the contrary that Aut(G, {a, a−1, b, b−1}) > ⟨α⟩. Then there would exist β ∈ Aut(G, {a, a−1, b, b−1}) such that aβ = b and bβ = a. Then cβ = [b, a] = c−1 and dβ = [b, c−1] = [c, b]c −1 = d−k. So (dk)β = d−k 2 = d since k2 ≡ −1(mod p). On the other hand, (dk)β = [bβ , cβ ] = [a, c−1] = d−1. It follows that d = d−1 which is impossible since d has order p > 2. Construction V Let p be an odd prime, and let m > n > 0 be integers. Take k ∈ Z∗p and t ∈ Z∗pn such that t2 ≡ −1(mod pn) and t ≡ k(mod p). Take j ∈ Zp. Let G(p,m, n; t, k, j) be a group with the following representation: ⟨a, b, c, d | ap m = cp = dp = 1, ap m−n = btp m−n dj , c = [a, b], [a, c] = d, [b, c] = dk, [a, d] = [b, d] = 1⟩ and let Θ(p,m, n; t, k, j) = BiCay(G(p,m, n; t, k, j), {a, a−1}, {b, b−1}, {1}). N. Li et al.: On cubic bi-Cayley graphs of p-groups 641 Lemma 4.5. The graph Θ(p,m, n; t, k, j) is a connected cubic non-Cayley vertex-transitive graph with 2p2m−n+2 vertices. Proof. Let G = G(p,m, n; t, k) and let Σ = Θ(p,m, n; t, k). Clearly, G′ = ⟨c⟩ × ⟨d⟩ ∼= Zp×Zp, d ∈ Z(G) and G/G′ = ⟨aG′, bG′⟩. Furthermore, G′∩⟨a⟩ = G′∩⟨b⟩ = 1. Since ap m−n = btp m−n dj , one has ⟨aG′⟩∩⟨bG′⟩ = ⟨apm−nG′⟩ ∼= Zpn , and so |G/G′| = p2m−n. It follows that G/G′ = ⟨aG′⟩ × ⟨ab−tG′⟩ ∼= Zpm × Zpm−n , and hence |G| = p2m−n+2. Thus, Σ has 2p2m−n+2 vertices. Let u = b, v = a−1, x = [u, v] and y = [u, x]. Note that x = [u, v] = [b, a−1] = [a, b]a −1 = aca−1c−1ac(ac)−1c = d(ac) −1 c = dc. Then y = [u, x] = [b, dc] = [b, c] = dk. It follows that xp = yp = 1. As ap m−n = btp m−n dj and t2 ≡ −1(mod pn), one has upm−n = bpm−n = a−tpm−ndjt = vtp m−n yj (due to t ≡ k(mod p)), and hence upm = 1. Furthermore, [v, x] = [a−1, c] = [c, a]a −1 = d−1 = dk 2 = yk. Clearly, [u, y] = [v, y] = 1 due to [a, d] = [b, d] = 1. Thus u, v, x, y have the same relations as do a, b, c, d. So G has an automorphism, say α, such that aα = b and bα = a−1, and hence α ∈ Aut(G, {a, a−1, b, b−1}). To complete the proof, by Proposition 3.4, it suffices to prove that Aut(G, {a, a−1, b, b−1}) = ⟨α⟩. Suppose on the contrary that Aut(G, {a, a−1, b, b−1}) > ⟨α⟩. Then there would exist β ∈ Aut(G, {a, a−1, b, b−1}) such that aβ = b and bβ = a. Then cβ = [b, a] = c−1 and dβ = [b, c−1] = [c, b]c −1 = d−k. So (dk)β = d−k 2 = d since k2 ≡ −1(mod p). On the other hand, (dk)β = [bβ , cβ ] = [a, c−1] = d−1. It follows that d = d−1 which is impossible since d has order p > 2. 5 Proof of Theorem 1.1 The goal of this section is to prove Theorem 1.1. Let G be a finite p-group with p a prime. By [18, 5.3.2], if |G : Φ(G)| = pr then every generating set of G has a subset of r elements which also generates G. In particular, G/Φ(G) ∼= Zrp. This implies that all minimal generating sets of a finite p-group G have the same cardinality, called the rank of G and denoted by d(G). Lemma 5.1. Let p be an odd prime and let H be a finite p-group such that d(H ′) ≤ 2. Let Γ be a connected cubic bi-Cayley of H . Let A ≤ Aut(Γ) be such that R(H) ≤ A. If Γ is a non-Cayley vertex-transitive graph and A is vertex- but not arc-transitive on Γ, then either p = 3 and d(H ′) = 2, or R(H)A. Proof. Assume that Γ is a non-Cayley vertex-transitive graph and A is vertex- but not arc-transitive on Γ. Assume further that R(H) is not normal in A. We shall prove that p = 3 and d(H ′) = 2. By Proposition 3.4, ΓM is a connected tetravalent Cayley graph of H , and by Lemma 3.3, ΓM is A-arc-transitive and R(H) ≤ A. For convenience, we may identify R(H) with H and let Σ = ΓM. Applying Proposition 2.2 to the A-arc- transitive Cayley graph Σ of odd order group H , we see that A has a normal subgroup R such that R ≤ H , Σ is a normal cover of ΣR, and moreover, ΣR and A/R satisfy 642 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 (1) or (2) of Proposition 2.2. Since Γ is non-symmetric, for each v ∈ V (Γ), Av is a 2- group, and so A is a {2, p}-group. It follows that A is solvable. If Proposition 2.2(1) happens, then A5 ≤ A/R ≤ S5, contradicting that A is solvable. If Proposition 2.2(2) happens, then H/R has two subgroups N/R and M/R such that H/R = N/R ⋊ M/R with M/R ∼= Zpm and N/R ∼= Znp , where n = pm. Then H ′ ≤ N . Since M ∩ N = R, one has M ∩H ′R = R, and so MH ′R/H ′R ∼= M/R ∼= Zpm . It follows that H/H ′R = N/H ′R×MH ′/H ′R. If N = H ′R, then H ′/(H ′∩R) ∼= N/R ∼= Znp . As d(H ′) ≤ 2, one has n = pm ≤ 2, contrary to that p is an odd prime. Thus, N > H ′R. By Proposition 3.4, we have d(H) ≤ 2, and so N/H ′R ∼= Zp. Then H ′/(H ′∩R) ∼= H ′R/R ∼= Zn−1p . Again, as d(H ′) ≤ 2, one has n − 1 = pm − 1 ≤ 2, and hence pm ≤ 3. Thus, d(H ′) = 2 and pm = 3. This completes the proof. We shall fulfill our task of the proof of Theorem 1.1 by the following three lemmas. Lemma 5.2. Let p > 3 be a prime and let H be a finite p-group such that either H ′ ∼= Zp × Zp or H ′ is cyclic. Let Γ be a connected cubic bi-Cayley of H . If Γ is a symmetric non-Cayley vertex-transitive graph, then Γ ∼= Γ(5, 1, 0, 0; 2) or Γ(5, 2; 2, 4) (see Construc- tions I & II). Proof. Assume that Γ is symmetric and non-Cayley. Since p > 3, by Proposition 2.3, we have p = 5. Let |H| = 5n. Then R(H) is a Sylow 5-subgroup of Aut(Γ). Let P be the maximal normal 5-subgroup of Aut(Γ). By [16, Lemma 18], we have |P | = 5n or 5n−1. If |P | = 5n, then P = R(H). Let Q be a Sylow 2-subgroup of Aut(Γ). Then A = R(H)⋊Q is vertex- but not arc-transitive on Γ. Since R(H)  A, by Proposition 3.4, we have Γ ∼= BiCay(H, {a, a−1}, {b, b−1}, {1}) with ⟨a, b⟩ = H . However, as R(H) = P  Aut(Γ) and since Γ is symmetric, the two orbits H0 and H1 of R(H) do not contain edges of Γ, a contradiction. If |P | = 5n−1, then by Proposition 2.6, Γ is a normal cover of the quotient graph ΓP of Γ relative to P , and Aut(Γ)/P is arc-transitive on ΓP . Clearly, ΓP has order 10. By inspecting the list of cubic symmetric graphs of order at most 10000 (see [6]), ΓP is isomorphic to the Petersen graph. It follows that Γ is not bipartite. So we have Γ = BiCay(H,R,L, S) with |R| = |L| > 0. By Proposition 2.4, we may assume that 1 ∈ S. Since |H| is odd and Γ has valency 3, one has R = {a, a−1} and L = {b, b−1}. Since Γ is connected, we have H = ⟨a, b⟩ by Proposition 2.4. Thus, d(H) ≤ 2. For convenience of the statement, in the following proof of this lemma, we shall identify R(H) with H . Clearly, P is maximal in H , so H ′ ≤ P . It follows that P/H ′ ≤ H/H ′ and P ′ ≤ H ′. If H ′ = 1, then by [11, Theorem 1.1] or [23, Proposition 5.1], we have H ∼= Z5 and Γ is just the Petersen graph. By Magma [3], Γ(5, 1, 0, 0; 2) is a symmetric cubic graph of order 10. So Γ ∼= Γ(5, 1, 0, 0; 2). Now assume that H ′ > 1. Then P > 1. Since P/H ′ ≤ H/H ′, one has d(P/H ′) ≤ 2. As P/H ′ ∼= (P/P ′)/(H ′/P ′) and since either H ′ ∼= Z5 × Z5 or H ′ is cyclic, one has d(P/P ′) ≤ 4. Since P ′ is characteristic in P and P  Aut(Γ), one has P ′  Aut(Γ). Consider the quotient graph ΓP ′ of Γ relative to P ′. By Proposition 2.6, Γ is a normal cover of ΓP ′ , and Aut(Γ)/P ′ is arc-transitive on ΓP ′ . As P/P ′ Aut(Γ)/P ′, the quotient graph, denote by ∆, of ΓP ′ relative to P/P ′ is isomorphic to ΓP and so isomorphic to the Petersen graph. So ΓP ′ is a normal P/P ′-cover of the Petersen graph. Note that P/P ′ is abelian. By [5, Theorem 7.1], we have P/P ′ ∼= Z5 × Z5 × Z5 as d(P/P ′) ≤ 4. Since d(P/H ′) ≤ 2, one has P ′ < H ′. Since either H ′ ∼= Z5 × Z5 or H ′ is cyclic, it follows N. Li et al.: On cubic bi-Cayley graphs of p-groups 643 that either H ′/P ′ ∼= Z5 or P ′ = 1 and H ′ ∼= Z5 × Z5. In the former case, H/P ′ is an inner abelian 5-group. Clearly, ΓP ′ is a bi-Cayley graph of H/P ′. By [16, Lemma 19] and [17, Lemma 4.7], we have H/P ′  Aut(ΓP ′), and hence H/P ′  Aut(Γ)/P ′. It follows that H  Aut(Γ), contrary to the maximality of P . For the latter case, we have P ′ = 1 and P ∼= Z5 × Z5 × Z5. So Γ is a cubic symmetric non-Cayley vertex-transitive graph of order 1250. By inspecting the list of cubic symmetric graphs of order at most 10000 (see [6]), up to isomorphism, there exists a unique cubic symmetric non-Cayley graph of order 1250, and by Magam [3], we see that Γ(5, 2; 2, 4) (see Construction II) is a cubic symmetric non-Cayley graph of order 1250. Thus, Γ ∼= Γ(5, 2; 2, 4). The next lemma will classify cubic non-Cayley vertex-transitive bi-Cayley graphs of a p-group H in case that H ′ is cyclic, where p > 3 is a prime. Lemma 5.3. Let p > 3 be a prime and let H be a finite p-group such that H ′ is cyclic. Let Γ = BiCay(H,R,L, S) be a connected cubic bi-Cayley of H . Then Γ is a non-Cayley vertex-transitive graph if and only if Γ is isomorphic to the graphs given in Construction I. Proof. From Lemma 4.1 we obtain the sufficiency. Next we prove the necessity. If Γ is symmetric, then by Lemma 5.2, we have Γ ∼= Γ(5, 1, 0, 0; 2). Now assume that Γ is not symmetric. Let A = Aut(Γ). By Lemma 5.1, we have R(H)A, and then by Proposition 3.4, we may let Γ = BiCay(H, {a, a−1}, {b, b−1}, {1}), and ΓM ∼= Cay(H, {a, b, a−1, b−1}), where H = ⟨a, b⟩. Furthermore, H has no au- tomorphisms interchanging the two pairs (a, a−1) and (b, b−1). As ΓM is A-arc-transitive by Lemma 3.3 and since R(H)  A, it follows that Aut(H, {a, b, a−1, b−1}) is transi- tive on {a, b, a−1, b−1}. Consequently, we have Aut(H, {a, b, a−1, b−1}) ∼= Z4. This implies that H has an automorphism, say α, such that aα = b and bα = a−1. Then [a, b]α = [b, a−1] = [a, b]a −1 and [a, b]α 2 = [a−1, b−1] = [a, b](ab) −1 . Since H = ⟨a, b⟩, one has H ′ = ⟨[a, b]h | h ∈ H⟩, and since d(H ′) = 1, one has H ′ = ⟨[a, b]⟩. It follows that [a, b]α = [a, b]k and [a, b]α2 = [a, b]k2 , where k is an integer coprime with p. Since α has order 4, one has k4 ≡ 1(mod |H ′|). On the other hand, as we already have [a, b]α = [a, b]a −1 . It follows that kp r ≡ 1(mod |H ′|), where pr is the order of a. Thus, k ≡ 1(mod |H ′|). This implies that [a, b] = [a, b]α = [a, b]a−1 and then [a, b] = [a, b]α 2 = [a−1, b−1] = [a, b](ab) −1 . Consequently, we obtain that [a, b] commutes with both a and b, and so [a, b] is contained in the center Z(H) of H . This implies that H ′ = ⟨[a, b]⟩. Assume that H/H ′ ∼= Zpm × Zpn with m ≥ n. Then ap m , bp m ∈ H ′. Let c = [a, b]. Assume that c has order ps. Since c ∈ Z(H), one has cpm = [apm , b] = 1 and hence m ≥ s. Suppose that apm = cpt with t ≤ s. Then bpm = (apm)α = (cpt)α = cpt = apm , implying that α fixes ap m . Since aα 2 = a−1, one has a−p m = ap m and so ap m = 1. Thus, ap m = bp m = 1. Then ⟨a⟩ ∩ ⟨c⟩ = ⟨b⟩ ∩ ⟨c⟩ = 1. As H/H ′ ∼= Zpm × Zpn , one has |⟨aH ′⟩⟨bH ′⟩| = pm+n. It follows that |⟨aH ′⟩ ∩ ⟨bH ′⟩| = pm−n, and hence ⟨apnH ′⟩ = ⟨bpnH ′⟩. Then apn = bℓpncλ, with ℓ ∈ Z∗pm−n and λ ∈ Zps . Now we have bp n = (ap n )α = (bℓp n cλ)α = a−ℓp n cλ. Then ap n = bℓp n cλ = a−ℓ 2pnc2λ. Since ⟨a⟩ ∩ ⟨c⟩ = 1, one has c2λ = 1 and hence cλ = 1. So apn = a−ℓ2pn . Hence, either m = n or ℓ2 ≡ −1(mod pm−n). If m = n, then we have H = ⟨a, b, c | ap m = bp m = cp s = 1, c = [a, b], [a, c] = [b, c] = 1⟩. 644 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 It is easy to see that β : a 7→ b, b 7→ a induces an automorphism of H . Clearly, β ∈ Aut(H, {a, b, a−1, b−1}). This, however, is impossible since Aut(H, {a, b, a−1, b−1} = ⟨α⟩ ∼= Z4. Now assume that m > n and let x = a, y = bℓ and z = [x, y]. Then z = cp ℓ , and H = H(p,m, n, s; t) = ⟨x, y, z | xp m = zp s = 1, z = [x, y], [x, z] =[y, z] = 1, xp n = yp n ⟩. Let t ∈ Z∗pm−n be such that tℓ ≡ 1(mod p m−n). Then b = yt and hence we obtain that Γ = BiCay(H, {x, x−1}, {yt, y−t}, {1}) = Γ(p,m, n, s; t). Finally, we shall classify cubic non-Cayley vertex-transitive bi-Cayley graphs of a p- group H in case H ′ ∼= Zp × Zp and p > 3 is prime. Lemma 5.4. Let p > 3 be a prime and let H be a finite p-group such that H ′ ∼= Zp × Zp. Let Γ = BiCay(H,R,L, S) be a connected cubic bi-Cayley of H . Then Γ is a non- Cayley vertex-transitive graph if and only if Γ is isomorphic to one of the graphs given Constructions II – V. Proof. The sufficiency can be obtained from Lemmas 4.2 – 4.5. In the following, we prove the necessity. Let A = Aut(Γ). If Γ is symmetric, then by Lemma 5.2, we have Γ ∼= Γ(5, 2; 2, 4) (see Construction II). Now assume that Γ is not symmetric. By Lemma 5.1, we have R(H)  A, and then by Proposition 3.4, we may let Γ = BiCay(H, {a, a−1}, {b, b−1}, {1}), and ΓM ∼= Cay(H, {a, b, a−1, b−1}), where H = ⟨a, b⟩. Furthermore, H has no automorphisms interchanging the two pairs (a, a−1) and (b, b−1). As ΓM is A-arc-transitive by Lemma 3.3 and since R(H)  A, it follows that Aut(H, {a, b, a−1, b−1}) is transitive on {a, b, a−1, b−1}. Consequently, we have Aut(H, {a, b, a−1, b−1}) ∼= Z4. This implies that H has an automorphism, say α, such that aα = b and bα = a−1. Since H ′ ∼= Zp × Zp and H ′ = ⟨[a, b]h | h ∈ H⟩, [a, b] has order p and [a, b] is not in the center of H . Let H3 = ⟨[[g, h], k]r | g, h, k ∈ {a, b}, r ∈ H⟩. Then H3 H . Since H is a p-group, one has H3 < H ′. So H3 ∼= Zp as H ′ ∼= Zp × Zp. Since H is a p-group, one has H3 ∩ Z(H) > 1, implying that H3 ≤ Z(H). As [a, b] is not in the center of H , one has H ′ ∩ Z(H) = H3. Let c = [a, b]. We shall first prove the following claim: Claim 1 There exists k ∈ Z∗p such that k2 ≡ −1(mod p) and [b, c] = [a, c]k. A direct computation shows that cα = ca −1 and cα 2 = c(ab) −1 . Using the fact that H3 ≤ Z(H), we obtain the following: [a, c]α = [b, ca −1 ] = [b, [a−1, c−1]c] = [b, c], [a, c]α 2 = [b, c]α = [a−1, ca −1 ] = [a−1, [a−1, c−1]c] = [a, c]−1, [a, c]α 3 = [b, c]−1. Then [a, c], [b, c] ̸= 1, and since H3 ∼= Zp, one has H3 = ⟨[a, c]⟩ = ⟨[b, c]⟩. As α is an automorphism of H of order 4, one has Hα3 = H3. We may let [a, c] α = [a, c]k with N. Li et al.: On cubic bi-Cayley graphs of p-groups 645 k ∈ Z∗p. Since [a, c]α = [b, c], we have [b, c] = [a, c]k, and since [a, c]α 2 = [a, c]−1, one has k2 ≡ −1(mod p), as claimed. Assume that ap m ∈ H ′ for some integer m > 0. Then bpm = (apm)α ∈ H ′. Then (H/H3) ′ = H ′/H3 = ⟨cH3⟩ ∼= Zp. Clearly, Φ(H ′) = 1. If H/H3 is metacyclic, then by [2, Theorem 2.3], H is metacyclic. This, however is impossible since H ′ ∼= Zp × Zp. Thus, H/H3 is not metacyclic. Since H ′/H3 ∼= Zp, it follows that H/H3 is inner abelian (see, for example, [2, Lemma 2.5]). Clearly, H/H3 = ⟨aH3⟩(⟨bH3, cH3⟩). Assume that ⟨aH3⟩ ∩ ⟨bH3, cH3⟩ ∼= Zpn with n < m. We shall consider the following two cases: Case 1: n = 0. In this case, H/H3 is generated by aH3, bH3, cH3 with the following defining rela- tions: ap m H3 = b pmH3 = c pH3 = H3, cH3 = [aH3, bH3], [cH3, aH3] = [cH3, bH3] = H3. If ⟨a⟩ ∩H3 ̸= 1, then a has order pm+1 and H3 = ⟨ap m⟩. Then [a, c] = aipm for some i ∈ Z∗p. As [a, c]α = [b, c] and aα = b, one has [b, c] = bip m . Again, since aα = b, b also has order pm+1 and H3 = ⟨bp m⟩. By Claim 1, we have apm = bkpm with k ∈ Z∗p and k2 ≡ −1(mod p). This implies that H is isomorphic to the following group: H(p,m; k, i) = ⟨a, b, c | ap m+1 = cp = 1, ap m = bkp m , c = [a, b], [a, c] = aip m , [b, c] = bip m ⟩. This is just the group given in Construction II. So Γ ∼= Γ(p,m; k, i). If ⟨a⟩∩H3 = 1, then a has order pm. Since aα = b, b also has order pm. Let d = [a, c]. By Claim 1, we have dα = [b, c] = dk with k ∈ Z∗p and k2 ≡ −1(mod p). This implies that H is isomorphic to the following group as given in Construction IV: ⟨a, b, c, d | ap m = bp m = cp = dp = 1, c = [a, b], [a, c] = d, [b, c] = dk, [a, d] = [b, d] = 1⟩. It follows that Γ ∼= ∆(p,m; k). Case 2: n > 0. In this case, we have ap m−n H3 = b tpm−ndH3 with t ∈ Z∗pn and d ∈ ⟨c⟩. Then dH3 = a pm−nb−tp m−n H3 ∈ ⟨ab−tH3⟩ as H/H3 is inner abelian. If dH3 ̸= H3, then we have H ′/H3 = ⟨dH3⟩ ≤ ⟨ab−tH3⟩ and then ⟨ab−tH3⟩H/H3. This implies that H/H3 is metacyclic since H/H3 = ⟨ab−tH3, bH3⟩. This is a contradiction. Thus, dH3 = H3 and so d = 1 as d ∈ ⟨c⟩. It follows that apm−nH3 = btp m−n H3 with t ∈ Z∗pn . In particular, ⟨apm−n , H3⟩ = ⟨bp m−n , H3⟩. Since aα = b and bα = a−1, it follows that bp m−n H3 = (a pm−nH3) α = (btp m−n H3) α = a−tp m−n H3 = b −t2pm−nH3. Consequently, we obtain that t2 ≡ −1(mod pn). Now we see that H/H3 is generated by aH3, bH3, cH3 with the following defining relations: bp m H3 = c pH3 = H3, a pm−nH3 = b tpm−nH3, cH3 = [aH3, bH3],[cH3, aH3] = [cH3, bH3] = H3. If ⟨a⟩ ∩H3 ̸= 1, then a has order pm+1 and H3 = ⟨ap m⟩. Since 0 < n < m, one has H3 ≤ ⟨ap m−n⟩ ∩ ⟨btpm−n⟩. As ⟨apm−n , H3⟩ = ⟨bp m−n , H3⟩, one has ⟨ap m−n⟩ = ⟨bpm−n⟩. 646 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 Then ap m−n = bsp m−n for some s ∈ Z∗pn . As (ap m−n )α = bp m−n and (bp m−n )α = a−p m−n , we have s2 ≡ −1(mod pn). Since H3 = ⟨[a, c]⟩, [a, c] = aip m for some i ∈ Z∗p. As [a, c]α = [b, c] and aα = b, one has [b, c] = bip m .This implies that H is isomorphic to the following group: H(p,m, n; s, i) = ⟨a, b, c | ap m+1 = cp = 1, ap m−n = bsp m−n , c = [a, b],[a, c] = aip m , [b, c] = bip m ⟩. This is just the group given in Construction III, and so Γ ∼= Γ(p,m, n; s, i). If ⟨a⟩ ∩ H3 = 1, then a has order pm. Since aα = b, b also has order pm. Let d = [a, c]. By Claim 1, we have dα = [b, c] = dk with k ∈ Z∗p and k2 ≡ −1(mod p). As ap m−n H3 = b tpm−nH3, one has ap m−n = btp m−n dj for some j ∈ Zp. Considering the image of ap m−n = btp m−n dj under α, we have t ≡ k(mod p). This implies that H is isomorphic to the following group which is just the group given in Construction V: G(p,m, n; t, k, j) = ⟨a, b, c, d | apm = cp = dp = 1, apm−n = btpm−ndj , c = [a, b], [a, c] = d, [b, c] = dk, [a, d] = [b, d] = 1⟩. Consequently, we obtain that Γ ∼= Θ(p,m, n; t, k, j). This completes the proof. 6 Proof of Theorem 1.2 In this final section, we shall prove Theorem 1.2 which gives a classification of cubic VNC- graphs of order p4 for each prime p. Proof of Theorem 1.2. By Lemmas 4.1, 4.2 and 4.4, we see that the graphs Γ(p,m, n, s; t) (m+ n+ 3 = 4,m > n ≥ 0, s ≥ 0), Γ(p, 1; k, i) and ∆(p, 1; k) are connected cubic non- Cayley vertex-transitive graphs of order 2p4. So we only need to prove the sufficiency. Let Γ be a connected cubic non-Cayley vertex-transitive graph of order 2p4. Assume first that Γ is symmetric. By [8, Corollary 3.4], every connected cubic symmetric graph of order 2pn is a Cayley graph whenever p ≥ 7 and n ≥ 1, and by [24, Theorem 1.2], every connected cubic symmetric graph order a 2-power is a Cayley graph. So we have p = 3 or 5. Then by inspecting the list of cubic symmetric graphs of order at most 10000 (see [6]), we obtain that p = 5, and up to isomorphism, there exists a unique cubic symmetric non-Cayley graph of order 1250. Thus, Γ ∼= Γ(5, 2; 2, 4) by Lemma 5.2. Assume now that Γ is not symmetric. If p ≤ 3, then by inspecting the list of cubic vertex-transitive graphs of order up to 1280 (see [15]), we see that every cubic vertex- transitive graph of order 32 or 162 is a Cayley graph, a contradiction. Thus, we may assume that p > 3. Since Γ is not symmetric, the stabilizer Aut(Γ)v of any vertex v ∈ V (Γ) in Aut(Γ) is a 2-group. Let P be a Sylow p-subgroup. Then P acts semiregularly on V (Γ) with two orbits. So Γ is a bi-Cayley graph of P . By Lemma 5.1, we have P  Aut(Γ), and then by Proposition 3.4, we may let Γ ∼= BiCay(P, {a, a−1}, {b, b−1}, {1}), where P = ⟨a, b⟩. Then |H ′| ≤ p2, and so either H ′ is cyclic or H ′ ∼= Z2p. By Lemmas 5.3 and 5.4, we have Γ is isomorphic to Γ(p,m, n, s; t)(m + n + 3 = 4,m > n ≥ 0, s ≥ 0), Γ(p, 1; k, i) or ∆(p, 1; k) since Γ has order 2p4. N. Li et al.: On cubic bi-Cayley graphs of p-groups 647 ORCID iDs Na Li https://orcid.org/0009-0001-4308-7543 Young Soo Kwon https://orcid.org/0000-0002-1765-0806 Jin-Xin Zhou https://orcid.org/0000-0002-8353-896X References [1] N. Biggs, Algebraic graph theory., Camb. Math. Libr., Cambridge University Press, Cam- bridge, 2nd edition, 1994, doi:10.1017/cbo9780511608704, https://doi.org/10. 1017/cbo9780511608704. [2] N. Blackburn, On prime-power groups with two generators, Proc. Camb. Philos. Soc. 54 (1958), 327–337, doi:10.1017/s0305004100033521, https://doi.org/10.1017/ s0305004100033521. [3] W. Bosma, J. Cannon and C. Playoust, The Magma algebra system. I: The user language, J. Symb. Comput. 24 (1997), 235–265, doi:10.1006/jsco.1996.0125, https://doi.org/10. 1006/jsco.1996.0125. [4] M. Conder and P. Dobcsányi, Trivalent symmetric graphs on up to 768 vertices, J. Comb. Math. Comb. Comput. 40 (2002), 41–63. [5] M. Conder and J. Ma, Arc-transitive abelian regular covers of cubic graphs, J. Algebra 387 (2013), 215–242, doi:10.1016/j.jalgebra.2013.02.035, https://doi.org/10.1016/j. jalgebra.2013.02.035. [6] Conder, Marston, Trivalent symmetric graphs on up to 10000 vertices, {https://www. math.auckland.ac.nz/˜conder/symmcubic10000list.txt}. [7] J.-L. Du, M. Conder and Y.-Q. Feng, Cubic core-free symmetric m-Cayley graphs, J. Al- gebr. Comb. 50 (2019), 143–163, doi:10.1007/s10801-018-0847-x, https://doi.org/ 10.1007/s10801-018-0847-x. [8] Y.-Q. Feng and J. H. Kwak, Cubic symmetric graphs of order twice an odd prime-power, J. Aust. Math. Soc. 81 (2006), 153–164, doi:10.1017/s1446788700015792, https://doi. org/10.1017/s1446788700015792. [9] C. D. Godsil, On the full automorphism group of a graph, Combinatorica 1 (1981), 243–256, doi:10.1007/bf02579330, https://doi.org/10.1007/bf02579330. [10] F. Gross, Conjugacy of odd order Hall subgroups, Bull. Lond. Math. Soc. 19 (1987), 311–319, doi:10.1112/blms/19.4.311, https://doi.org/10.1112/blms/19.4.311. [11] I. Kovács, A. Malnič, D. Marušič and Š. Miklavič, One-matching bi-Cayley graphs over Abelian groups, Eur. J. Comb. 30 (2009), 602–616, doi:10.1016/j.ejc.2008.06.001, https: //doi.org/10.1016/j.ejc.2008.06.001. [12] C. H. Li and J. Pan, Finite 2-arc-transitive abelian Cayley graphs, Eur. J. Comb. 29 (2008), 148–158, doi:10.1016/j.ejc.2006.12.001, https://doi.org/10.1016/j.ejc.2006. 12.001. [13] D. Marušič, On vertex symmetric digraphs, Discrete Math. 36 (1981), 69–81, doi:10.1016/ 0012-365x(81)90174-6, https://doi.org/10.1016/0012-365x(81)90174-6. [14] P. Potočnik, P. Spiga and G. Verret, Cubic vertex-transitive graphs on up to 1280 vertices, J. Symb. Comput. 50 (2013), 465–477, doi:10.1016/j.jsc.2012.09.002, https://doi.org/ 10.1016/j.jsc.2012.09.002. [15] P. Potočnik, P. Spiga and G. Verret, A census of small connected cubic vertex-transitive graphs, 2021, http://www.matapp.unimib.it/spiga/. 648 Ars Math. Contemp. 23 (2023) #P4.09 / 631–648 [16] Y.-L. Qin and J.-X. Zhou, Cubic edge-transitive bi-p-metacirculants, Electron. J. Comb. 25 (2018), research paper p3.28, 19, doi:10.37236/6417, https://doi.org/10.37236/ 6417. [17] Y.-L. Qin and J.-X. Zhou, Cubic edge-transitive bi-Cayley graphs over inner-abelian p-groups, Commun. Algebra 47 (2019), 1973–1984, doi:10.1080/00927872.2018.1527919, https:// doi.org/10.1080/00927872.2018.1527919. [18] D. J. S. Robinson, A course in the theory of groups., volume 80 of Grad. Texts Math., Springer- Verlag, New York, NY, 2nd edition, 1995. [19] M.-M. Zhang and J.-X. Zhou, Trivalent vertex-transitive bi-dihedrants, Discrete Math. 340 (2017), 1757–1772, doi:10.1016/j.disc.2017.03.017, https://doi.org/10.1016/j. disc.2017.03.017. [20] M.-M. Zhang and J.-X. Zhou, Trivalent dihedrants and bi-dihedrants, Ars Math. Contemp. 21 (2021), 26, doi:10.26493/1855-3974.2373.c02, id/No 2, https://doi.org/10.26493/ 1855-3974.2373.c02. [21] J. X. Zhou, Cubic vertex-transitive graphs of order 2p2, Adv. Math., Beijing 37 (2008), 605– 609. [22] J.-X. Zhou, Tetravalent half-arc-transitive p-graphs, J. Algebr. Comb. 44 (2016), 947–971, doi: 10.1007/s10801-016-0696-4, https://doi.org/10.1007/s10801-016-0696-4. [23] J.-X. Zhou and Y.-Q. Feng, Cubic bi-Cayley graphs over abelian groups, Eur. J. Comb. 36 (2014), 679–693, doi:10.1016/j.ejc.2013.10.005, https://doi.org/10.1016/j.ejc. 2013.10.005. [24] J.-X. Zhou and Y.-Q. Feng, The automorphisms of bi-Cayley graphs, J. Comb. Theory, Ser. B 116 (2016), 504–532, doi:10.1016/j.jctb.2015.10.004, https://doi.org/10.1016/j. jctb.2015.10.004. [25] J.-X. Zhou and Y.-Q. Feng, Cubic non-Cayley vertex-transitive bi-Cayley graphs over a regular p-group, Electron. J. Comb. 23 (2016), research paper p3.34, 10, doi:10.37236/3915, https: //doi.org/10.37236/3915. ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 23 (2023) #P4.10 / 649–682 https://doi.org/10.26493/1855-3974.2948.f25 (Also available at http://amc-journal.eu) Using a q-shuffle algebra to describe the basic module V (Λ0) for the quantized enveloping algebra Uq(ŝl2)* Paul Terwilliger † Department of Mathematics, University of Wisconsin 480 Lincoln Drive, Madison, WI 53706-1388 USA Received 25 August 2022, accepted 25 April 2023, published online 26 June 2023 Abstract We consider the quantized enveloping algebra Uq(ŝl2) and its basic module V (Λ0). This module is infinite-dimensional, irreducible, integrable, and highest-weight. We de- scribe V (Λ0) using a q-shuffle algebra in the following way. Start with the free associative algebra V on two generators x, y. The standard basis for V consists of the words in x, y. In 1995 M. Rosso introduced an associative algebra structure on V, called a q-shuffle algebra. For u, v ∈ {x, y} their q-shuffle product is u ⋆ v = uv + q(u,v)vu, where (u, v) = 2 (resp. (u, v) = −2) if u = v (resp. u ̸= v). Let U denote the subalgebra of the q-shuffle algebra V that is generated by x, y. Rosso showed that the algebra U is isomorphic to the positive part of Uq(ŝl2). In our first main result, we turn U into a Uq(ŝl2)-module. Let U denote the Uq(ŝl2)-submodule of U generated by the empty word. In our second main result, we show that the Uq(ŝl2)-modules U and V (Λ0) are isomorphic. Let V denote the subspace of V spanned by the words that do not begin with y or xx. In our third main result, we show that U = U ∩V. Keywords: Quantized enveloping algebra, q-Serre relations, basic module, q-shuffle algebra. Math. Subj. Class. (2020): 17B37, 05E14, 81R50 *No funding was received for conducting this study. All data generated or analysed during this study are included in this published article. †The author thanks Pascal Baseilhac for many conversations about Uq(ŝl2) and U+q . The author has no relevant financial or non-financial interests to disclose. E-mail address: terwilli@math.wisc.edu (Paul Terwilliger) cb This work is licensed under https://creativecommons.org/licenses/by/4.0/ 650 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 1 Introduction The quantized enveloping algebra Uq(ŝl2) is associative, noncommutative, and infinite di- mensional. A presentation by generators and relations is given in Appendix B below. The algebra Uq(ŝl2) has a subalgebra U+q , called the positive part. The algebra U + q has a pre- sentation involving two generators A,B and two relations, called the q-Serre relations: [A, [A, [A,B]q]q−1 ] = 0, [B, [B, [B,A]q]q−1 ] = 0. Both U+q and Uq(ŝl2) are well known in algebraic combinatorics [4,21,26], representation theory [11,12,15,47], and mathematical physics [6,7,17,27]. In the following paragraphs, we describe a few situations in which U+q and Uq(ŝl2) play a role. There is an object in algebraic combinatorics called a tridiagonal pair [21]. Roughly speak- ing, this is a pair of diagonalizable linear maps on a finite-dimensional vector space, that each act on the eigenspaces of the other one in a block-tridiagonal fashion. According to [21, Example 1.7], for a finite-dimensional irreducible U+q -module V on which the gen- erators A,B are not nilpotent, the pair A,B act on V as a tridiagonal pair. The resulting tridiagonal pair is said to have q-geometric type or q-Serre type. This type of tridiagonal pair is described in [1–3, 22–26, 36, 48]. Another object in algebraic combinatorics is a partially ordered set Y called the Young lattice [35, page 288]. The elements of Y are the Young diagrams (partitions), and the partial order is given by diagram inclusion. Define a vector space V consisting of the formal linear combinations of Y . In the Hayashi realization [4, Theorem 10.6] the vector space V becomes an integrable Uq(ŝl2)-module with the following features. Each Young diagram λ is a weight vector. The Uq(ŝl2)-generators E0, E1, F0, F1 act on λ as follows. Color the boxes of λ alternating blue and red, with the top left box colored blue. The generator E0 (resp. E1) sends λ to a linear combination of the Young diagrams µ obtained from λ by removing a blue box (resp. red box). In this linear combination the µ-coefficient is a power of q that depends on the location of the box λ/µ. Similarly, F0 (resp. F1) sends λ to a linear combination of the Young diagrams µ obtained from λ by adding a blue box (resp. red box). In this linear combination the µ-coefficient is a power of q that depends on the location of the box µ/λ. The Uq(ŝl2)-submodule of V generated by the empty Young diagram is denoted V (Λ0) and called the basic representation. The Uq(ŝl2)-module V (Λ0) is infinite-dimensional, irreducible, integrable, and highest-weight. For more detail about V (Λ0) see [4, Chapter 10], [20, Section 9], [27, Chapter 5]. Next we recall an embedding, due to M. Rosso [33, 34] of U+q into a q-shuffle algebra. Start with a free associative algebra V on two generators x, y. These generators are called letters. For n ≥ 0, a word of length n in V is a product of letters ℓ1ℓ2 · · · ℓn. We interpret the word of length 0 to be the multiplicative identity in V; this word is called trivial and denoted by 1. The words in V form a basis for the vector space V; this basis is called standard. In [33, 34] M. Rosso introduced an associative algebra structure on V, called a q-shuffle algebra. For letters u, v their q-shuffle product is u ⋆ v = uv + q(u,v)vu, where (u, v) = 2 (resp. (u, v) = −2) if u = v (resp. u ̸= v). In [34, Theorem 15] Rosso gave an injective algebra homomorphism ♮ from U+q into the q-shuffle algebra V, that sends A 7→ x and B 7→ y. We mention some applications of the map ♮ : U+q → V. In [16] I. Damiani obtained a Poincaré-Birkhoff-Witt (or PBW) basis for U+q whose elements are defined recursively. In [11, Proposition 6.1] J. Beck obtained another PBW basis for U+q by adjusting some of P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 651 the elements in the Damiani PBW basis. In [40] (resp. [45]) we applied the map ♮ to the Damiani (resp. Beck) PBW basis, and expressed the images in the standard basis for V. We gave the images in closed form [40, Theorem 1.7], [45, Theorem 7.1]. The map ♮ is used in [39] to define the alternating elements of U+q . In [39, Theorem 10.1] a set of alternating elements is shown to form a PBW basis for U+q . This PBW basis is said to be alternating [39, Definition 10.3]. In [38] we used the alternating elements to obtain a central extension U+q of U+q . The algebra U+q is defined by generators and relations. These generators, said to be alternating, are in bijection with the alternating elements of U+q . By [38, Lemma 3.3] there exists a surjective algebra homomorphism U+q → U+q that sends each alternating generator of U+q to the corresponding alternating element in U+q . In [38, Lemma 3.6] this homomorphism is adjusted to obtain an algebra isomorphism U+q → U+q ⊗ F[z1, z2, . . .] where F is the ground field and {zn}∞n=1 are mutually commuting indeterminates. By [38, Theorem 10.2] the alternating generators form a PBW basis for U+q . The algebra U+q is called the alternating central extension of U+q [38, 46]. We remark that U+q is related to the work of Baseilhac, Koizumi, Shigechi concerning the q-Onsager algebra and integrable lattice models [8, 10]. See [5–7, 9, 37, 41–44, 46] for related work. Turning to the present paper, our goal is to describe the basic Uq(ŝl2)-module V (Λ0) using the q-shuffle algebra V. We have three main results, which are summarized below. Let End(V) denote the algebra consisting of the linear maps from V to V. We now define some maps X,Y,K in End(V). The map X (resp. Y ) is the automorphism of the free algebra V that sends x 7→ qx and y 7→ y (resp. x 7→ x and y 7→ qy). Define K = X2Y −2. Define the maps A∗L, B ∗ L, A ∗ R, B ∗ R in End(V) that send 1 7→ 0 and for a nontrivial word w = ℓ1ℓ2 · · · ℓn in V, A∗Lw = ℓ2 · · · ℓnδℓ1,x, B∗Lw = ℓ2 · · · ℓnδℓ1,y, A∗Rw = ℓ1 · · · ℓn−1δℓn,x, B∗Rw = ℓ1 · · · ℓn−1δℓn,y. Here δr,s is the Kronecker delta. Define the maps Aℓ, Bℓ, Ar, Br in End(V) such that for v ∈ V, Aℓv = x ⋆ v, Bℓv = y ⋆ v, Arv = v ⋆ x, Brv = v ⋆ y. Let U denote the subalgebra of the q-shuffle algebra V that is generated by x, y. By con- struction the map ♮ : U+q → U is an algebra isomorphism. Our first main result is that U becomes a Uq(ŝl2)-module on which the Uq(ŝl2)-generators act as follows: generator E0 F0 K±10 E1 F1 K ±1 1 D ±1 action on U A∗R qArK −1−q−1Aℓ q−q−1 q ±1K∓1 B∗R BrK−Bℓ q−q−1 K ±1 X∓1 Let U denote the submodule of the Uq(ŝl2)-module U that is generated by the vector 1. Our second main result is that the Uq(ŝl2)-modules U and V (Λ0) are isomorphic. Let V denote the intersection of the kernel of B∗L and the kernel of (A ∗ L) 2. The vector space V has a basis consisting of the words in V that do not begin with y or xx. Note that the sum V = F1+ Fx+ xyV is direct. Our third main result is that U = U ∩V. The paper is organized as follows. Section 2 contains some preliminaries. In Section 3 we recall the algebra U+q and discuss its basic properties. In Section 4 we describe the free algebra V. In Section 5 we describe the maps X,Y,K in End(V). In Section 6 we 652 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 describe the maps A∗L, B ∗ L, A ∗ R, B ∗ R in End(V). In Section 7 we describe the q-shuffle algebra V. In Section 8 we describe the maps Aℓ, Bℓ, Ar, Br in End(V). In Section 9 we describe the subalgebra U of the q-shuffle algebra V. In Sections 10, 11 we give our main results, which are Theorems 10.1, 10.7, 11.11. In Section 12 we describe some variations on Theorem 10.1. In Appendix A we display some relations that are satisfied by the maps from the main body of the paper. In Appendix B we give a presentation of Uq(ŝl2). In Appendix C we give a basis for some of the weight spaces of the Uq(ŝl2)-module U. In Appendix D we show how the Uq(ŝl2)-generators act on the bases in Appendix C. In Appendix E we discuss a linear algebraic situation that comes up in Section 11. 2 Preliminaries We now begin our formal argument. Recall the natural numbers N = {0, 1, 2, . . .} and integers Z = {0,±1,±2, . . .}. Let F denote a field with characteristic zero. Throughout this paper, every vector space we discuss is understood to be over F. Every algebra we discuss is understood to be associative, over F, and have a multiplicative identity. A subal- gebra has the same multiplicative identity as the parent algebra. Let A denote an algebra. An automorphism of A is an algebra isomorphism A → A. The opposite algebra Aopp consists of the vector space A and the multiplication map A × A → A, (a, b) 7→ ba. An antiautomorphism of A is an algebra isomorphism A → Aopp. We recall a few concepts from linear algebra. Let V denote a vector space, and consider an F-linear map T : V → V . The map T is said to be nilpotent whenever there exists a positive integer n such that Tn = 0. The map T is said to be locally nilpotent whenever for all v ∈ V there exists a positive integer n such that Tnv = 0. If T is nilpotent then T is locally nilpotent. If T is locally nilpotent and the dimension of V is finite, then T is nilpotent. Throughout the paper, fix a nonzero q ∈ F that is not a root of unity. Recall the notation [n]q = qn − q−n q − q−1 n ∈ N. 3 The positive part of Uq(ŝl2) Later in the paper, we will discuss the quantized enveloping algebra Uq(ŝl2). For now, we consider a subalgebra U+q of Uq(ŝl2), called the positive part. Shortly we will give a presentation of U+q by generators and relations. For elements X ,Y in any algebra, define their commutator and q-commutator by [X ,Y] = XY − YX , [X ,Y]q = qXY − q−1YX . Note that [X , [X , [X ,Y]q]q−1 ] = X 3Y − [3]qX 2YX + [3]qXYX 2 − YX 3. (3.1) Definition 3.1 (See [30, Corollary 3.2.6]). Define the algebra U+q by generators A, B and relations [A, [A, [A,B]q]q−1 ] = 0, [B, [B, [B,A]q]q−1 ] = 0. (3.2) P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 653 We call U+q the positive part of Uq(ŝl2). The relations (3.2) are called the q-Serre relations. We mention some symmetries of U+q . Lemma 3.2. There exists an automorphism σ of U+q that sends A ↔ B. Moreover σ2 = id, where id denotes the identity map. Lemma 3.3 (See [40, Lemma 2.2]). There exists an antiautomorphism † of U+q that fixes each of A, B. Moreover †2 = id. Lemma 3.4 (See [41, Lemma 3.4]). The maps σ, † commute. Definition 3.5. Let τ denote the composition of σ and †. Note that τ is an antiautomor- phism of U+q that sends A ↔ B. We have τ2 = id. Next we describe a grading for the algebra U+q . The q-Serre relations are homogeneous in both A and B. Therefore, the algebra U+q has a N2-grading for which A and B are homogeneous, with degrees (1, 0) and (0, 1) respectively. For (r, s) ∈ N2 let U+q (r, s) denote the (r, s)-homogeneous component of the grading. The dimension of U+q (r, s) is described by a generating function, as we now discuss. Let t and u denote commuting indeterminates. Definition 3.6. Define the generating function Φ(t, u) = ∞∏ n=1 1 1− tnun−1 1 1− tnun 1 1− tn−1un . Using (1− z)−1 = 1 + z + z2 + · · · we expand the above generating function as a power series: Φ(t, u) = ∑ (r,s)∈N2 dr,st rus, dr,s ∈ N. For notational convenience, define dr,−1 = 0 and d−1,s = 0 for r, s ∈ N. Example 3.7 (See [39, Example 3.4]). For 0 ≤ r, s ≤ 6 we display dr,s in the (r, s)-entry of the matrix below:  1 1 1 1 1 1 1 1 2 3 3 3 3 3 1 3 6 8 9 9 9 1 3 8 14 19 21 22 1 3 9 19 32 42 48 1 3 9 21 42 66 87 1 3 9 22 48 87 134  We have Φ(t, u) = Φ(u, t). Moreover dr,s = ds,r for (r, s) ∈ N2. Lemma 3.8 (See [39, Definition 3.2, Corollary 3.7]). For (r, s) ∈ N2 we have dr,s = dimU + q (r, s). 654 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 Our next goal is to show that dr,s−1 ≤ dr,s and dr−1,s ≤ dr,s for (r, s) ∈ N2. To reach the goal, we modify the generating function Φ(t, u) in the following way. Definition 3.9. Define the generating function ∆(t, u) = ∞∏ n=1 1 1− tnun−1 1 1− tnun 1 1− tnun+1 . (3.3) Lemma 3.10. We have ∆(t, u) = Φ(t, u)(1− u) and ∆(u, t) = Φ(t, u)(1− t). Moreover ∆(t, u) = ∑ (r,s)∈N2 ( dr,s − dr,s−1 ) trus, ∆(u, t) = ∑ (r,s)∈N2 ( dr,s − dr−1,s ) trus. Proof. Use Definitions 3.6, 3.9. Lemma 3.11. For (r, s) ∈ N2 we have dr,s−1 ≤ dr,s and dr−1,s ≤ dr,s. Proof. Expand the right-hand side of (3.3) as a power series. In this power series, the coeffi- cient of trus is nonnegative for (r, s) ∈ N2. The result follows in view of Lemma 3.10. Our next general goal is to compute max{dr,s|s ∈ N} for r ∈ N, and max{dr,s|r ∈ N} for s ∈ N. To reach the goal, we will use the concept of a partition. For n ∈ N, a partition of n is a sequence λ = {λi}∞i=1 of natural numbers such that λi ≥ λi+1 for i ≥ 1 and n = ∑∞ i=1 λi. Let pn denote the number of partitions of n. For example, n 0 1 2 3 4 5 6 pn 1 1 2 3 5 7 11 Define the generating function for partitions: p(t) = ∑ n∈N pnt n. (3.4) The following result is well known; see for example [13, Theorem 8.3.4]. p(t) = ∞∏ n=1 1 1− tn . (3.5) We expand the generating function ( p(t) )3 as a power series:( p(t) )3 = ∑ n∈N µnt n, µn ∈ N. (3.6) Consider the coefficients {µn}n∈N. For example, n 0 1 2 3 4 5 6 µn 1 3 9 22 51 108 221 P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 655 Proposition 3.12. For r ∈ N we have µr = max{dr,s|s ∈ N}. (3.7) For s ∈ N we have µs = max{dr,s|r ∈ N}. (3.8) Proof. First, for r ∈ N we verify (3.7). Let µ′r denote right-hand side of (3.7). We show that µr = µ′r. By Lemma 3.11, we may view µ′r = ∑ s∈N (dr,s − dr,s−1). By this and Lemma 3.10, ∆(t, 1) = ∑ (r,s)∈N2 ( dr,s − dr,s−1 ) tr = ∑ r∈N µ′rt r. (3.9) Set u = 1 in (3.3), and evaluate the result using (3.5), (3.6). This yields ∆(t, 1) = ∞∏ n=1 1 (1− tn)3 = ( p(t) )3 = ∑ r∈N µrt r. (3.10) Comparing (3.9), (3.10) we obtain µr = µ′r for r ∈ N. We have verified (3.7). The second assertion in the proposition statement follows from the first assertion in the proposition statement and the comment above Lemma 3.8. Our next general goal is to embed U+q into a q-shuffle algebra. For this q-shuffle algebra the underlying vector space is a free algebra on two generators. This free algebra is described in the next section. 4 The free algebra V Let x, y denote noncommuting indeterminates. Let V denote the free algebra with genera- tors x, y. By a letter in V we mean x or y. For n ∈ N, a word of length n in V is a product of letters ℓ1ℓ2 · · · ℓn. We interpret the word of length 0 to be the multiplicative identity in V; this word is called trivial and denoted by 1. The vector space V has a basis consisting of its words; this basis is called standard. We mention some symmetries of the free algebra V. For the next four lemmas, the proofs are routine and omitted. Lemma 4.1. There exists an automorphism σ of the free algebra V that sends x ↔ y. Moreover σ2 = id. Lemma 4.2. There exists an antiautomorphism † of the free algebra V that fixes each of x, y. Moreover †2 = id. Lemma 4.3. The map σ from Lemma 4.1 commutes with the map † from Lemma 4.2. 656 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 Lemma 4.4. There exists an antiautomorphism τ of the free algebra V that sends x ↔ y. The map τ is the composition the map σ from Lemma 4.1 and the map † from Lemma 4.2. We have τ2 = id. Example 4.5. The automorphism σ sends xxx ↔ yyy, xxyy ↔ yyxx, xyxxyy ↔ yxyyxx. The antiautomorphism † sends xxx ↔ xxx, xxyy ↔ yyxx, xyxxyy ↔ yyxxyx. The antiautomorphism τ sends xxx ↔ yyy, xxyy ↔ xxyy, xyxxyy ↔ xxyyxy. The free algebra V has a N2-grading for which x and y are homogeneous, with degrees (1, 0) and (0, 1) respectively. For (r, s) ∈ N2 let V(r, s) denote the (r, s)-homogeneous component of the grading. These homogeneous components are described as follows. Let w = ℓ1ℓ2 · · · ℓn denote a word in V. The x-degree of w is the cardinality of the set {i|1 ≤ i ≤ n, ℓi = x}. The y-degree of w is the cardinality of the set {i|1 ≤ i ≤ n, ℓi = y}. For (r, s) ∈ N2 the subspace V(r, s) has a basis consisting of the words in V that have x-degree r and y-degree s. The dimension of V(r, s) is equal to the binomial coefficient ( r+s r ) . By construction V(0, 0) = F1. By construction, the sum V = ∑ (r,s)∈N2 V(r, s) is direct. Example 4.6. The following is a basis for the vector space V(2, 3): xxyyy, xyxyy, xyyxy, xyyyx, yxxyy, yxyxy, yxyyx, yyxxy, yyxyx, yyyxx. Let End(V) denote the algebra consisting of the F-linear maps from V to V. Let I denote the identity in End(V). 5 The maps X , Y , K In this section we describe some maps X , Y , K in End(V) that will be used in our main results. Definition 5.1. Let X denote the automorphism of the free algebra V that sends x 7→ qx and y 7→ y. Let Y denote the automorphism of the free algebra V that sends x 7→ x and y 7→ qy. Example 5.2. The map X sends xxx 7→ q3xxx, xxyy 7→ q2xxyy, xyxxyy 7→ q3xyxxyy. The map Y sends xxx 7→ xxx, xxyy 7→ q2xxyy, xyxxyy 7→ q3xyxxyy. Lemma 5.3. For (r, s) ∈ N2 the maps X and Y act on V(r, s) as qrI and qsI , respectively. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 657 Proof. By the description of V(r, s) above Example 4.6. By construction the maps X , Y are invertible, and they commute. Definition 5.4. Define K = X2Y −2. Thus K is the automorphism of the free algebra V that sends x 7→ q2x and y 7→ q−2y. Example 5.5. The map K sends xxx 7→ q6xxx, xxyy 7→ xxyy, xyxxyy 7→ xyxxyy. Lemma 5.6. For (r, s) ∈ N2 the map K acts on V(r, s) as q2r−2sI . Proof. By Lemma 5.3 and Definition 5.4. Lemma 5.7. The following diagrams commute: V X ±1 −−−−→ V σ y yσ V −−−−→ Y ±1 V V Y ±1 −−−−→ V σ y yσ V −−−−→ X±1 V V K ±1 −−−−→ V σ y yσ V −−−−→ K∓1 V V X ±1 −−−−→ V † y y† V −−−−→ X±1 V V Y ±1 −−−−→ V † y y† V −−−−→ Y ±1 V V K ±1 −−−−→ V † y y† V −−−−→ K±1 V V X ±1 −−−−→ V τ y yτ V −−−−→ Y ±1 V V Y ±1 −−−−→ V τ y yτ V −−−−→ X±1 V V K ±1 −−−−→ V τ y yτ V −−−−→ K∓1 V Proof. Routine. 6 The maps A∗L, B ∗ L, A ∗ R, B ∗ R In this section we recall from [32] some maps A∗L, B ∗ L, A ∗ R, B ∗ R in End(V) that will be used in our main results. First we mention some notation. The Kronecker delta δr,s is equal to 1 if r = s, and 0 if r ̸= s. Definition 6.1 (See [32, Lemma 4.3]). Define the maps A∗L, B∗L, A∗R, B∗R in End(V) as follows. For a nontrivial word w = ℓ1ℓ2 · · · ℓn in V, A∗Lw = ℓ2 · · · ℓnδℓ1,x, B∗Lw = ℓ2 · · · ℓnδℓ1,y, A∗Rw = ℓ1 · · · ℓn−1δℓn,x, B∗Rw = ℓ1 · · · ℓn−1δℓn,y. Moreover A∗L1 = 0, B ∗ L1 = 0, A ∗ R1 = 0, B ∗ R1 = 0. (6.1) 658 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 Example 6.2. The maps A∗L, B∗L, A∗R, B∗R are illustrated in the table below. w x y xx xy yx yy A∗Lw 1 0 x y 0 0 B∗Lw 0 1 0 0 x y A∗Rw 1 0 x 0 y 0 B∗Rw 0 1 0 x 0 y Lemma 6.3. For v ∈ V, A∗L(xv) = v, A ∗ L(yv) = 0, B ∗ L(xv) = 0, B ∗ L(yv) = v, A∗R(vx) = v, A ∗ R(vy) = 0, B ∗ R(vx) = 0, B ∗ R(vy) = v. Proof. Use Definition 6.1. For notational convenience, define V(r,−1) = 0 and V(−1, s) = 0 for r, s ∈ N. Lemma 6.4. For (r, s) ∈ N2 we have A∗LV(r, s) ⊆ V(r − 1, s), B∗LV(r, s) ⊆ V(r, s− 1), A∗RV(r, s) ⊆ V(r − 1, s), B∗RV(r, s) ⊆ V(r, s− 1). Proof. By Definition 6.1 or Lemma 6.3. Lemma 6.5. The maps A∗L, B∗L, A∗R, B∗R are locally nilpotent on the vector space V. Proof. We mentioned above Example 4.6 that the sum V = ∑ (r,s)∈N2 V(r, s) is direct. The result follows from this and Lemma 6.4. Next we describe how the maps X , Y are related to the maps A∗L, B ∗ L, A ∗ R, B ∗ R. Lemma 6.6. We have XA∗L = q −1A∗LX, XB ∗ L = B ∗ LX, XA ∗ R = q −1A∗RX, XB ∗ R = B ∗ RX, Y A∗L = A ∗ LY, Y B ∗ L = q −1B∗LY, Y A ∗ R = A ∗ RY, Y B ∗ R = q −1B∗RY. Proof. By Lemmas 5.3, 6.4. The next result is about A∗L and B ∗ L; a similar result holds for A ∗ R and B ∗ R. Observe that the sum V = F1+ xV+ yV is direct. Lemma 6.7. The following (i) – (v) hold: (i) kerA∗L has a basis consisting of the words in V that do not begin with x; (ii) kerA∗L = F1+ yV; (iii) kerB∗L has a basis consisting of the words in V that do not begin with y; (iv) kerB∗L = F1+ xV; (v) kerA∗L ∩ kerB∗L = F1. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 659 Proof. Use Definition 6.1 and the observation above the lemma statement. The following result appears in [32]; we give a short proof for the sake of completeness. Lemma 6.8 (See [32, Lemma 4.6]). Let W denote a nonzero subspace of V that is closed under A∗L and B ∗ L. Then 1 ∈ W . Proof. For n ∈ N define Vn = ∑ r+s≤n V(r, s). Note that V0 = F1. We have Vn−1 ⊆ Vn for n ≥ 1, and V = ∪n∈NVn. For n ≥ 1 we have A∗LVn ⊆ Vn−1 and B∗LVn ⊆ Vn−1, in view of Lemma 6.4. Since W ̸= 0, there exists n ∈ N such that W ∩ Vn ̸= 0. Assume for the moment that n = 0. Then 1 ∈ W and we are done. Next assume that n ≥ 1. Without loss, we may assume that W ∩ Vn−1 = 0. Pick 0 ̸= v ∈ W ∩ Vn. We have A∗Lv ∈ W ∩ Vn−1 = 0 and B∗Lv ∈ W ∩ Vn−1 = 0, so v ∈ F1 in view of Lemma 6.7(v). By construction 0 ̸= v ∈ W , so 1 ∈ W . Lemma 6.9. Let W denote a nonzero subspace of V that is closed under A∗R and B∗R. Then 1 ∈ W . Proof. The †-image W † is a subspace of V that is invariant under A∗L and B∗L. We have 1 ∈ W † by Lemma 6.8, and 1† = 1 by construction, so 1 ∈ W . Lemma 6.10. The following diagrams commute: V A∗L−−−−→ V σ y yσ V −−−−→ B∗L V V B∗L−−−−→ V σ y yσ V −−−−→ A∗L V V A∗R−−−−→ V σ y yσ V −−−−→ B∗R V V B∗R−−−−→ V σ y yσ V −−−−→ A∗R V V A∗L−−−−→ V † y y† V −−−−→ A∗R V V B∗L−−−−→ V † y y† V −−−−→ B∗R V V A∗R−−−−→ V † y y† V −−−−→ A∗L V V B∗R−−−−→ V † y y† V −−−−→ B∗L V V A∗L−−−−→ V τ y yτ V −−−−→ B∗R V V B∗L−−−−→ V τ y yτ V −−−−→ A∗R V V A∗R−−−−→ V τ y yτ V −−−−→ B∗L V V B∗R−−−−→ V τ y yτ V −−−−→ A∗L V Proof. Routine. 7 The q-shuffle algebra V In the previous sections we discussed the free algebra V. There is another algebra structure on V, called the q-shuffle algebra. This algebra was introduced by Rosso [33, 34] and described further by Green [18]. We will adopt the approach of [18], which is suited to our purpose. The q-shuffle product is denoted by ⋆. To describe this product, we first consider some special cases. We have 1 ⋆ v = v ⋆ 1 = v for v ∈ V. For letters u, v we have u ⋆ v = uv + vuq(u,v), where 660 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 ( , ) x y x 2 −2 y −2 2 Thus x ⋆ x = (1 + q2)xx, x ⋆ y = xy + q−2yx, y ⋆ x = yx+ q−2xy, y ⋆ y = (1 + q2)yy. For a letter u and a nontrivial word v = v1v2 · · · vn in V, u ⋆ v = n∑ i=0 v1 · · · viuvi+1 · · · vnq(v1,u)+(v2,u)+···+(vi,u), (7.1) v ⋆ u = n∑ i=0 v1 · · · viuvi+1 · · · vnq(vn,u)+(vn−1,u)+···+(vi+1,u). (7.2) For example y ⋆ (xxx) = yxxx+ q−2xyxx+ q−4xxyx+ q−6xxxy, (xxx) ⋆ y = q−6yxxx+ q−4xyxx+ q−2xxyx+ xxxy. For nontrivial words u = u1u2 · · ·ur and v = v1v2 · · · vs in V, u ⋆ v = u1 ( (u2 · · ·ur) ⋆ v ) + v1 ( u ⋆ (v2 · · · vs) ) q(u1,v1)+(u2,v1)+···+(ur,v1), u ⋆ v = ( u ⋆ (v1 · · · vs−1) ) vs + ( (u1 · · ·ur−1) ⋆ v ) urq (ur,v1)+(ur,v2)+···+(ur,vs). For example, assume r = 2 and s = 2. Then u ⋆ v = u1u2v1v2 + u1v1u2v2q (u2,v1) + u1v1v2u2q (u2,v1)+(u2,v2) + v1u1u2v2q (u1,v1)+(u2,v1) + v1u1v2u2q (u1,v1)+(u2,v1)+(u2,v2) + v1v2u1u2q (u1,v1)+(u1,v2)+(u2,v1)+(u2,v2). The map σ from Lemma 4.1 is an automorphism of the q-shuffle algebra V. The map † from Lemma 4.2 is an antiautomorphism of the q-shuffle algebra V. The map τ from Lemma 4.4 is an antiautomorphism of the q-shuffle algebra V. Above Example 4.6 we mentioned an N2-grading of the free algebra V. This is also an N2-grading for the q-shuffle algebra V. See [19, 29, 31, 32, 38–40] for more information about the q-shuffle algebra V. 8 The maps Aℓ, Bℓ, Ar, Br In this section we recall from [32] some maps Aℓ, Bℓ, Ar, Br in End(V) that will be used in our main results. Definition 8.1 (See [32, Definition 7.1]). Define the maps Aℓ, Bℓ, Ar, Br in End(V) as follows. For v ∈ V, Aℓv = x ⋆ v, Bℓv = y ⋆ v, Arv = v ⋆ x, Brv = v ⋆ y. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 661 Example 8.2. The maps Aℓ, Bℓ, Ar, Br are illustrated in the table below. v 1 x y xy Aℓv x q[2]qxx xy + q −2yx q[2]qxxy + xyx Bℓv y q −2xy + yx q[2]qyy q −1[2]qxyy + yxy Arv x q[2]qxx q −2xy + yx q−1[2]qxxy + xyx Brv y xy + q −2yx q[2]qyy q[2]qxyy + yxy Lemma 8.3. For (r, s) ∈ N2 we have AℓV(r, s) ⊆ V(r + 1, s), BℓV(r, s) ⊆ V(r, s+ 1), ArV(r, s) ⊆ V(r + 1, s), BrV(r, s) ⊆ V(r, s+ 1). Proof. By Definition 8.1 and the description of V(r, s) above Example 4.6. Next we describe how the maps X , Y are related to the maps Aℓ, Bℓ, Ar, Br. Lemma 8.4. We have XAℓ = qAℓX, XBℓ = BℓX, XAr = qArX, XBr = BrX, Y Aℓ = AℓY, Y Bℓ = qBℓY, Y Ar = ArY, Y Br = qBrY. Proof. By Lemmas 5.3, 8.3. Lemma 8.5. The following diagrams commute: V Aℓ−−−−→ V σ y yσ V −−−−→ Bℓ V V Bℓ−−−−→ V σ y yσ V −−−−→ Aℓ V V Ar−−−−→ V σ y yσ V −−−−→ Br V V Br−−−−→ V σ y yσ V −−−−→ Ar V V Aℓ−−−−→ V † y y† V −−−−→ Ar V V Bℓ−−−−→ V † y y† V −−−−→ Br V V Ar−−−−→ V † y y† V −−−−→ Aℓ V V Br−−−−→ V † y y† V −−−−→ Bℓ V V Aℓ−−−−→ V τ y yτ V −−−−→ Br V V Bℓ−−−−→ V τ y yτ V −−−−→ Ar V V Ar−−−−→ V τ y yτ V −−−−→ Bℓ V V Br−−−−→ V τ y yτ V −−−−→ Aℓ V Proof. Routine. 662 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 9 The subspace U In this section we discuss a subspace U ⊆ V that will be used in our main results. Definition 9.1. Let U denote the subalgebra of the q-shuffle algebra V that is generated by x, y. The algebra U is described as follows. By [33, Theorem 13] or [18, page 10], x ⋆ x ⋆ x ⋆ y − [3]qx ⋆ x ⋆ y ⋆ x+ [3]qx ⋆ y ⋆ x ⋆ x− y ⋆ x ⋆ x ⋆ x = 0, y ⋆ y ⋆ y ⋆ x− [3]qy ⋆ y ⋆ x ⋆ y + [3]qy ⋆ x ⋆ y ⋆ y − x ⋆ y ⋆ y ⋆ y = 0. So in the q-shuffle algebra V the elements x, y satisfy the q-Serre relations. Therefore, there exists an algebra homomorphism ♮ from U+q to the q-shuffle algebra V, that sends A 7→ x and B 7→ y. The map ♮ has image U by Definition 9.1, and is injective by [34, Theorem 15]. Consequently ♮ : U+q → U is an algebra isomorphism. By construction the following diagrams commute: U+q ♮−−−−→ V σ y yσ U+q −−−−→ ♮ V U+q ♮−−−−→ V † y y† U+q −−−−→ ♮ V U+q ♮−−−−→ V τ y yτ U+q −−−−→ ♮ V (9.1) Consequently U is invariant under each of σ, †, τ . Earlier we mentioned an N2-grading for both the algebra U+q and the q-shuffle algebra V. These gradings are related as follows. The algebra U has an N2-grading inherited from U+q via ♮. With respect to this grad- ing, for (r, s) ∈ N2 the (r, s)-homogeneous component of U is the ♮-image of the (r, s)- homogeneous component of U+q . We denote this homogeneous component by U(r, s). By construction, U(r, s) = V(r, s) ∩ U, (r, s) ∈ N2. (9.2) By construction U(0, 0) = F1. By construction, the sum U = ∑ (r,s)∈N2 U(r, s) is direct. By Lemma 3.8 and the construction, dr,s = dimU(r, s), (r, s) ∈ N2. (9.3) Lemma 9.2. For (r, s) ∈ N2 the following hold on U(r, s): X = qrI, Y = qsI, K = q2r−2sI. Proof. By Lemmas 5.3, 5.6 and since U(r, s) ⊆ V(r, s). Lemma 9.3. The vector space U is invariant under each of X±1, Y ±1, K±1. Proof. By Lemma 9.2 and since U = ∑ (r,s)∈N2 U(r, s). P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 663 Lemma 9.4 (See [32, Proposition 9.1]). The vector space U is invariant under each of A∗L, B ∗ L, A ∗ R, B ∗ R. For notational convenience, define U(r,−1) = 0 and U(−1, s) = 0 for r, s ∈ N. Lemma 9.5. For (r, s) ∈ N2 we have A∗LU(r, s) ⊆ U(r − 1, s), B∗LU(r, s) ⊆ U(r, s− 1), A∗RU(r, s) ⊆ U(r − 1, s), B∗RU(r, s) ⊆ U(r, s− 1). Proof. By (9.2) and Lemmas 6.4, 9.4. Lemma 9.6. The subspace U is invariant under each of Aℓ, Bℓ, Ar, Br. Proof. By Definitions 9.1, 8.1. Lemma 9.7. For (r, s) ∈ N2 we have AℓU(r, s) ⊆ U(r + 1, s), BℓU(r, s) ⊆ U(r, s+ 1), ArU(r, s) ⊆ U(r + 1, s), BrU(r, s) ⊆ U(r, s+ 1). Proof. By (9.2) and Lemmas 8.3, 9.6. In [32, Propositions 9.1, 9.3] there are many relations satisfied by the maps K, K−1, A∗L, B ∗ L, A ∗ R, B ∗ R, Aℓ, Bℓ, Ar, Br. For convenience we reproduce these relations in Appendix A. These relations will be used in our main results. 10 The Uq(ŝl2)-module U and its submodule U We now bring in the algebra Uq(ŝl2). The definition of this algebra can be found in Ap- pendix B. In the present section, we turn the vector space U into a Uq(ŝl2)-module, and describe the submodule U generated by the vector 1. The following is our first main result. Theorem 10.1. The vector space U becomes a Uq(ŝl2)-module on which the Uq(ŝl2)- generators act as follows: generator E0 F0 K±10 E1 F1 K ±1 1 D ±1 action on U A∗R qArK −1−q−1Aℓ q−q−1 q ±1K∓1 B∗R BrK−Bℓ q−q−1 K ±1 X∓1 Proof. This is routinely checked using Lemmas 9.3, 9.4, 9.6 along with the relations in Lemmas 6.6, 8.4 and Appendix A. Among the things to check, is that qArK−1−q−1Aℓ and BrK−Bℓ satisfy the q-Serre relations. This can be checked easily using [32, Lemma 10.3, Corollary 10.4]. 664 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 Consider the Uq(ŝl2)-module U from Theorem 10.1. Recall the N2-grading of U from around (9.2). Next we describe how the Uq(ŝl2)-generators act on the homogeneous com- ponents of this grading. Lemma 10.2. For (r, s) ∈ N2 the following hold on U(r, s): K0 = q 2s−2r+1I, K1 = q 2r−2sI, D = q−rI. (10.1) Moreover E0U(r, s) ⊆ U(r − 1, s), F0U(r, s) ⊆ U(r + 1, s), (10.2) E1U(r, s) ⊆ U(r, s− 1), F1U(r, s) ⊆ U(r, s+ 1). (10.3) Proof. By Lemmas 9.2, 9.5, 9.7 and the data in Theorem 10.1. In this paragraph we recall a few concepts about Uq(ŝl2)-modules; see for example [20, Section 3.2]. Let W denote a Uq(ŝl2)-module. A weight space for W is a common eigenspace for the action of K0,K1, D on W . The sum of these weight spaces is direct. We call W a weight module whenever W is equal to the sum of its weight spaces. If W is a weight module, then every submodule of W is a weight module [20, Proposition 3.2.1]. We return our attention to the Uq(ŝl2)-module U from Theorem 10.1. We mentioned above Lemma 9.2 that the sum U = ∑ (r,s)∈N2 U(r, s) is direct. By this and (10.1), the Uq(ŝl2)- module U is a weight module, and its weight spaces are the nonzero subspaces among U(r, s) (r, s ∈ N). Note that these weight spaces have finite dimension. It turns out that the Uq(ŝl2)-module U is not irreducible. Next we consider its submodules. Definition 10.3. Let U denote the submodule of the Uq(ŝl2)-module U that is generated by the vector 1. Lemma 10.4. For the Uq(ŝl2)-module U, (i) U is contained in every nonzero submodule of the Uq(ŝl2)-module U; (ii) U is the unique irreducible submodule of the Uq(ŝl2)-module U. Proof. (i) Let W denote a nonzero submodule of the Uq(ŝl2)-module U. The vector space W is invariant under A∗R, B ∗ R by Theorem 10.1, so 1 ∈ W by Lemma 6.9. The Uq(ŝl2)- module U is generated by 1, so U ⊆ W . (ii) By (i) above. Next we consider how the Uq(ŝl2)-generators act on the vector 1. Lemma 10.5. For the Uq(ŝl2)-module U, K01 = q1, K11 = 1, D1 = 1, E01 = 0, F 2 0 1 = 0, E11 = 0, F11 = 0. Proof. This is routinely checked using the data in Theorem 10.1 and Lemma 10.2. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 665 There is a well known Uq(ŝl2)-module V (Λ0) that is said to be basic; see [20, page 221]. The module V (Λ0) is highest weight, integrable, and level one; see [4, Chapter 10] and [27, Chapter 5]. The module V (Λ0) is characterized as follows. Lemma 10.6 (See [27, pages 63, 64]). There exists a Uq(ŝl2)-module V (Λ0) with the following property: V (Λ0) is generated by a nonzero vector v such that K0v = qv, K1v = v, Dv = v, E0v = 0, F 2 0 v = 0, E1v = 0, F1v = 0. Moreover V (Λ0) is irreducible, infinite-dimensional, and unique up to isomorphism of Uq(ŝl2)-modules. The following is our second main result. Theorem 10.7. The Uq(ŝl2)-modules U and V (Λ0) are isomorphic. Proof. By Definition 10.3 and Lemmas 10.5, 10.6. Descriptions of V (Λ0) can be found in [4, Chapter 10] and [20, Section 9] and [27, Chap- ter 5]; see also [14, Section 20.4] and [28, Chapter 14]. Our next general goal is to describe V (Λ0) from the point of view of U. Definition 10.8. For (r, s) ∈ N2 define U(r, s) = U ∩ U(r, s). The Uq(ŝl2)-module U is a weight module, and its weight spaces are the nonzero subspaces among U(r, s) (r, s ∈ N). More detail is given in the next result. Lemma 10.9. The following (i) – (iv) hold for the Uq(ŝl2)-module U. (i) U(0, 0) = F1. (ii) The sum U = ∑ (r,s)∈N2 U(r, s) is direct. (iii) For (r, s) ∈ N2 the following hold on U(r, s): K0 = q 2s−2r+1I, K1 = q 2r−2sI, D = q−rI. (iv) For (r, s) ∈ N2, E0U(r, s) ⊆ U(r − 1, s), F0U(r, s) ⊆ U(r + 1, s), E1U(r, s) ⊆ U(r, s− 1), F1U(r, s) ⊆ U(r, s+ 1), where U(r,−1) = 0 and U(−1, s) = 0. Proof. (i) By Definition 10.8 and since U(0, 0) = F1. (ii) Since U is a weight module. (iii), (iv) By Lemma 10.2 and Definition 10.8. Our next goal is to describe the weight space dimensions for the Uq(ŝl2)-module U. Recall the partition numbers {pn}n∈N from Section 3. 666 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 Proposition 10.10. For (r, s) ∈ N2 the vector space U(r, s) ̸= 0 if and only if r ≥ (r−s)2. In this case dimU(r, s) = pn, where n = r − (r − s)2. Proof. For the Uq(ŝl2)-module V (Λ0) the weight space dimensions are described in [20, pages 221, 222]. The result follows from that description and Theorem 10.7 above. Example 10.11. For 0 ≤ r, s ≤ 6 the dimension of U(r, s) is given in the (r, s)-entry of the matrix below:  1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 1 2 1 0 0 0 0 0 2 3 2 0 0 0 0 1 3 5 3 1 0 0 0 1 5 7 5 0 0 0 0 2 7 11  Compare the above matrix with the one in Example 3.7. Next we describe the generating function ∑ (r,s)∈N2 dimU(r, s)t rus. Define the generating function ϕ(t, u) = ∑ n∈Z tn 2 un 2−n. (10.4) Note that ϕ(t, u) = 1 + t+ tu2 + t4u2 + t4u6 + · · · . Proposition 10.12. We have∑ (r,s)∈N2 dimU(r, s)trus = p(tu)ϕ(t, u), where p(t) is from (3.4) and ϕ(t, u) is from (10.4). Proof. This is a reformulation of Proposition 10.10. In Appendix C, we give a basis for each nonzero U(r, s) such that r + s ≤ 10. 11 A characterization of the Uq(ŝl2)-module U In order to motivate this section, we glance at the basis vectors displayed in Appendix C. Each displayed vector is a linear combination of some words in V that do not begin with y or xx. Consequently, each displayed vector is contained in the kernel of B∗L and the kernel of (A∗L) 2. Using this observation, we will characterize the Uq(ŝl2)-module U. Definition 11.1. Let V denote the intersection of the kernel of B∗L and the kernel of (A∗L)2. Note that V is a subspace of the vector space V. We have several comments about V. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 667 Lemma 11.2. The vector space V has a basis consisting of the words in V that do not begin with y or xx. Proof. By Definitions 6.1, 11.1. Lemma 11.3. The sum V = F1+ Fx+ xyV is direct. Proof. By Lemma 11.2. We are going to show that U = U ∩V. We will do this in several steps. In the first step, we show that U ∩V is a submodule of the Uq(ŝl2)-module U. Lemma 11.4. The vector space V is invariant under each of X±1, Y ±1, K±1, A∗R, B ∗ R. Proof. Use Definitions 5.1, 5.4 and Lemma 11.3. Lemma 11.5. The vector space V is invariant under each of qArK −1 − q−1Aℓ, BrK −Bℓ. Proof. We will use Lemma 11.3. The map qArK−1 − q−1Aℓ sends 1 7→ (q − q−1)x and x 7→ 0. The map BrK −Bℓ sends 1 7→ 0 and x 7→ q2x ⋆ y − y ⋆ x = q2(xy + q−2yx)− (yx+ q−2xy) = (q2 − q−2)xy. Pick (r, s) ∈ N2 and a word w ∈ V(r, s). The map qArK−1 − q−1Aℓ sends xyw 7→ q1+2s−2r(xyw) ⋆ x− q−1x ⋆ (xyw). (11.1) Using (7.2) we obtain (xyw) ⋆ x = xy(w ⋆ x) + [2]qq 2r−2s−1xxyw. (11.2) Using (7.1) we obtain x ⋆ (xyw) = q[2]qxxyw + xy(x ⋆ w). (11.3) By (11.1) – (11.3) the map qArK−1 − q−1Aℓ sends xyw 7→ xy ( q1+2s−2rw ⋆ x− q−1x ⋆ w ) . The map BrK −Bℓ sends xyw 7→ q2r−2s(xyw) ⋆ y − y ⋆ (xyw). (11.4) Using (7.2) we obtain (xyw) ⋆ y = xy(w ⋆ y) + q2s−2r+2xyyw + q2s−2ryxyw. (11.5) Using (7.1) we obtain y ⋆ (xyw) = yxyw + q−2xyyw + xy(y ⋆ w). (11.6) By (11.4) – (11.6) the map BrK −Bℓ sends xyw 7→ xy ( (q2 − q−2)yw + q2r−2sw ⋆ y − y ⋆ w ) . The result follows from the above comments. 668 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 Lemma 11.6. The vector space U ∩V is a submodule of the Uq(ŝl2)-module U. Proof. By Theorem 10.1 and Lemmas 11.4, 11.5. Lemma 11.7. The vector space U is a submodule of the Uq(ŝl2)-module U ∩V. Proof. We have 1 ∈ U by the comment below (9.2). We have 1 ∈ V by Lemma 11.3. So 1 ∈ U ∩V. The result follows in view of Definition 10.3 and Lemma 11.6. The Uq(ŝl2)-module U ∩ V is a weight module, and its weight spaces are the nonzero subspaces among U(r, s) ∩ V (r, s ∈ N). We will return to the Uq(ŝl2)-module U ∩ V after some comments about U. Lemma 11.8. The Uq(ŝl2)-generators E0, E1 are locally nilpotent on the Uq(ŝl2)-module U. Proof. By Lemma 6.5 and Theorem 10.1. Lemma 11.9. The Uq(ŝl2)-generators F0, F1 are not locally nilpotent on the Uq(ŝl2)- module U. Proof. The words xx and y are contained in U, but Fn0 (xx) ̸= 0 and Fn1 y ̸= 0 for all n ∈ N. Lemma 11.10. The Uq(ŝl2)-generators F0, F1 are locally nilpotent on the Uq(ŝl2)-module U ∩V. Proof. First consider F0. Assume that there exists v ∈ U ∩V such that Fn+10 v ̸= 0 for all n ∈ N. We will get a contradiction. By our comments below Lemma 11.7, we may assume without loss of generality that v ∈ U(r, s) for some (r, s) ∈ N2. We have r ≥ 1 and s ≥ 1, by Lemma 11.3 and F 20 1 = 0 and F0x = 0. Let n ∈ N. By (10.2) and Lemma 11.6, we have Fn+10 v ∈ U(r + n + 1, s) ∩ V. In particular F n+1 0 v ∈ V, so (A∗L)2F n+1 0 v = 0 and B∗LF n+1 0 v = 0 in view of Definition 11.1. We have A ∗ LF n+1 0 v ̸= 0, by Lemma 6.7(v) and since B∗LF n+1 0 v = 0. We have A ∗ LF n+1 0 v ∈ U(r + n, s) by Lemma 9.5. By these comments 0 ̸= A∗LF n+1 0 v ∈ ker(A∗L)∩U(r+n, s). Therefore 0 ̸= ker(A∗L)∩U(r+n, s). The map A∗L is locally nilpotent by Lemma 6.5. In Appendix A we find A ∗ LAℓ−q2AℓA∗L = I . The vector space U is invariant under A∗L and Aℓ. By these comments, we may apply Appendix E with S = A∗L and T = Aℓ and V = U. By Lemma 17.8 the map A∗L is surjective on U. By this and Lemma 9.5, A∗LU(r + n, s) = U(r + n − 1, s). By this and 0 ̸= ker(A∗L) ∩ U(r + n, s), we obtain dimU(r + n − 1, s) < dimU(r + n, s). Since n ∈ N is arbitrary, dimU(r − 1, s) < dimU(r, s) < dimU(r + 1, s) < dimU(r + 2, s) < · · · This contradicts (3.8) and (9.3), so F0 is locally nilpotent. Next we consider F1. Assume that there exists v ∈ U ∩ V such that Fm+11 v ̸= 0 for all m ∈ N. We will get a contradiction. By our comments below Lemma 11.7, we may assume without loss of generality that v ∈ U(r, s) for some (r, s) ∈ N2. We have r ≥ 1 and s ≥ 1, by Lemma 11.3 and F11 = 0 and F 31 x = 0. Let m ∈ N. By (10.3) and Lemma 11.6, we have Fm+11 v ∈ U(r, s + m + 1) ∩ V. In particular F m+1 1 v ∈ V, so B∗LF m+1 1 v = 0 in view of Definition 11.1. By these comments 0 ̸= Fm+11 v ∈ ker(B∗L) ∩ U(r, s +m + 1). P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 669 Therefore 0 ̸= ker(B∗L)∩U(r, s+m+1). The map B∗L is locally nilpotent by Lemma 6.5. In Appendix A we find B∗LBℓ − q2BℓB∗L = I . The vector space U is invariant under B∗L and Bℓ. By these comments, we may apply Appendix E with S = B ∗ L and T = Bℓ and V = U. By Lemma 17.8 the map B∗L is surjective on U. By this and Lemma 9.5, B∗LU(r, s+m+1) = U(r, s+m). By this and 0 ̸= ker(B∗L)∩U(r, s+m+1), we obtain dimU(r, s+m) < dimU(r, s+m+ 1). Since m ∈ N is arbitrary, dimU(r, s) < dimU(r, s+ 1) < dimU(r, s+ 2) < dimU(r, s+ 3) < · · · This contradicts (3.7) and (9.3), so F1 is locally nilpotent. The following is our third main result. Theorem 11.11. We have U = U ∩V. Proof. By Lemmas 11.8, 11.10 the Uq(ŝl2)-generators E0, E1, F0, F1 are locally nilpotent on the Uq(ŝl2)-module U ∩V. Therefore, the Uq(ŝl2)-module U ∩V is integrable in the sense of [4, Definition 4.2]. By this and [20, Theorem 3.5.4], the Uq(ŝl2)-module U∩V is completely reducible. By Lemma 11.7, U is a submodule of the Uq(ŝl2)-module U ∩ V. By these comments, there exists a submodule W of the Uq(ŝl2)-module U ∩ V such that the sum U ∩V = U +W is direct. Assume for the moment that W ̸= 0. Then U ⊆ W by Lemma 10.4(i), for a contradiction. Consequently W = 0, so U = U ∩V. 12 Variations on the theme In Theorem 10.1 we turned the vector space U into a Uq(ŝl2)-module. In this section we describe three more ways to do this. Each way yields a Uq(ŝl2)-module U that is isomorphic to the one in Theorem 10.1. Proposition 12.1. For each row in the table below, the vector space U becomes a Uq(ŝl2)- module on which the Uq(ŝl2)-generators act as indicated: generator E0 F0 K±10 E1 F1 K ±1 1 D ±1 action on U B∗R qBrK−q−1Bℓ q−q−1 q ±1K±1 A∗R ArK −1−Aℓ q−q−1 K ∓1 Y ∓1 action on U A∗L qAℓK −1−q−1Ar q−q−1 q ±1K∓1 B∗L BℓK−Br q−q−1 K ±1 X∓1 action on U B∗L qBℓK−q−1Br q−q−1 q ±1K±1 A∗L AℓK −1−Ar q−q−1 K ∓1 Y ∓1 The above three Uq(ŝl2)-modules U are isomorphic to the Uq(ŝl2)-module U in Theo- rem 10.1. For row 1 (resp. row 2) (resp. row 3), a Uq(ŝl2)-module isomorphism is given by the restriction of σ (resp. †) (resp. τ ) to U. Proof. Below (9.1) we mentioned that U is invariant under each of σ, †, τ . The result follows from this along with Lemmas 9.3, 9.4, 9.6 and Lemmas 5.7, 6.10, 8.5. 13 Appendix A: Some relations In this appendix we list some relations satisfied by the maps K, K−1, A∗L, B ∗ L, A ∗ R, B ∗ R, Aℓ, Bℓ, Ar, Br. 670 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 Proposition 13.1 (See [32, Proposition 9.1]). We have KA∗L = q −2A∗LK, KB ∗ L = q 2B∗LK, KA∗R = q −2A∗RK, KB ∗ R = q 2B∗RK, KAℓ = q 2AℓK, KBℓ = q −2BℓK, KAr = q 2ArK, KBr = q −2BrK, A∗LA ∗ R = A ∗ RA ∗ L, B ∗ LB ∗ R = B ∗ RB ∗ L, A∗LB ∗ R = B ∗ RA ∗ L, B ∗ LA ∗ R = A ∗ RB ∗ L, AℓAr = ArAℓ, BℓBr = BrBℓ, AℓBr = BrAℓ, BℓAr = ArBℓ, A∗LBr = BrA ∗ L, B ∗ LAr = ArB ∗ L, A∗RBℓ = BℓA ∗ R, B ∗ RAℓ = AℓB ∗ R, A∗LBℓ = q −2BℓA ∗ L, B ∗ LAℓ = q −2AℓB ∗ L, A∗RBr = q −2BrA ∗ R, B ∗ RAr = q −2ArB ∗ R, A∗LAℓ − q2AℓA∗L = I, A∗RAr − q2ArA∗R = I, B∗LBℓ − q2BℓB∗L = I, B∗RBr − q2BrB∗R = I, A∗LAr −ArA∗L = K, B∗LBr −BrB∗L = K−1, A∗RAℓ −AℓA∗R = K, B∗RBℓ −BℓB∗R = K−1, A3ℓBℓ − [3]qA2ℓBℓAℓ + [3]qAℓBℓA2ℓ −BℓA3ℓ = 0, B3ℓAℓ − [3]qB2ℓAℓBℓ + [3]qBℓAℓB2ℓ −AℓB3ℓ = 0, A3rBr − [3]qA2rBrAr + [3]qArBrA2r −BrA3r = 0, B3rAr − [3]qB2rArBr + [3]qBrArB2r −ArB3r = 0. Proposition 13.2 (See [32, Proposition 9.3]). The following relations hold on U: (A∗L) 3B∗L − [3]q(A∗L)2B∗LA∗L + [3]qA∗LB∗L(A∗L)2 −B∗L(A∗L)3 = 0, (B∗L) 3A∗L − [3]q(B∗L)2A∗LB∗L + [3]qB∗LA∗L(B∗L)2 −A∗L(B∗L)3 = 0, (A∗R) 3B∗R − [3]q(A∗R)2B∗RA∗R + [3]qA∗RB∗R(A∗R)2 −B∗R(A∗R)3 = 0, (B∗R) 3A∗R − [3]q(B∗R)2A∗RB∗R + [3]qB∗RA∗R(B∗R)2 −A∗R(B∗R)3 = 0. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 671 14 Appendix B: The algebra Uq(ŝl2) In this appendix we recall the quantized enveloping algebra Uq(ŝl2). We will generally follow the approach of Ariki [4, Section 3.3]. We will refer to the matrix A = ( 2 −2 −2 2 ) . We index the rows and columns of A by 0, 1. Definition 14.1 (See [4, Definition 3.16]). Define the algebra Uq(ŝl2) by generators K±1i , D ±1, Ei, Fi, i ∈ {0, 1} and the following relations. For i, j ∈ {0, 1}, KiK −1 i = K −1 i Ki = 1, DD −1 = D−1D = 1, [Ki,Kj ] = 0, [D,Ki] = 0, KiEjK −1 i = q Ai,jEj , KiFjK −1 i = q −Ai,jFj , DE0D −1 = qE0, DF0D −1 = q−1F0, [D,E1] = 0, [D,F1] = 0, [Ei, Fj ] = δi,j Ki −K−1i q − q−1 , [Ei, [Ei, [Ei, Ej ]q]q−1 ] = 0, [Fi, [Fi, [Fi, Fj ]q]q−1 ] = 0, i ̸= j. Note 14.2. The Ariki notation is related to our notation as follows. Ariki notation our notation v q ti Ki vd D αj(hi) Ai,j Note 14.3 (See [4, Section 3.3]). The algebra Uq(ŝl2) is sometimes called the quantum algebra of type A(1)1 . 15 Appendix C: The subspaces U(r, s) In this appendix, we give a basis for each nonzero U(r, s) such that r + s ≤ 10. r s basis for U(r, s) 0 0 1 r s basis for U(r, s) 1 0 x r s basis for U(r, s) 1 1 xy 672 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 r s basis for U(r, s) 2 1 xyx 1 2 xyy r s basis for U(r, s) 2 2 xyxy, xyyx r s basis for U(r, s) 3 2 xyxyx, xyyxx 2 3 xyxyy + xyyxy r s basis for U(r, s) 4 2 xyxyxx+ [3]qxyyxxx 3 3 xyxyxy, xyyxxy, xyyxyx+ xyxyyx r s basis for U(r, s) 4 3 xyxyxyx, xyyxyxx+ xyxyyxx+ xyyxxyx, xyxyxxy + [3]qxyyxxxy + xyyxxyx 3 4 xyyxxyy, xyyxyxy + xyxyyxy + xyxyxyy r s basis for U(r, s) 5 3 [3]qxyyxyxxx+ [3]qxyxyyxxx+ [2] 2 qxyyxxyxx +2xyxyxyxx+ xyxyxxyx+ [3]qxyyxxxyx 4 4 xyxyxyxy, xyyxxyyx, xyyxyxxy + xyxyyxxy + xyyxxyxy, xyxyxxyy + [3]qxyyxxxyy + xyyxxyxy, xyyxyxyx+ xyxyyxyx+ xyxyxyyx r s basis for U(r, s) 5 4 [3]qxyyxyxxxy + [3]qxyxyyxxxy + [2] 2 qxyyxxyxxy +2xyxyxyxxy + xyxyxxyxy + [3]qxyyxxxyxy +xyyxyxxyx+ xyxyyxxyx+ xyyxxyxyx, xyxyxyxyx, xyyxxyyxx, xyxyxxyyx+ [3]qxyyxxxyyx+ xyyxxyxyx, xyyxyxyxx+ xyxyyxyxx+ xyxyxyyxx +xyyxyxxyx+ xyxyyxxyx+ xyyxxyxyx 4 5 xyyxyxxyy + xyxyyxxyy + xyyxxyxyy + xyyxxyyxy, xyxyxyyxy + xyxyyxyxy + xyyxyxyxy + xyxyxyxyy, [3]qxyxyxxyyy + [3] 2 qxyyxxxyyy + [3]qxyyxxyxyy + xyxyxyxyy P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 673 r s basis for U(r, s) 6 4 [3]qxyyxyxxxyx+ [3]qxyxyyxxxyx+ [2] 2 qxyyxxyxxyx +2xyxyxyxxyx+ xyxyxxyxyx+ [3]qxyyxxxyxyx +xyyxyxxyxx+ xyxyyxxyxx+ xyyxxyxyxx +3xyxyxyxyxx+ [3]qxyyxyxyxxx+ [3]qxyxyyxyxxx +[3]qxyxyxyyxxx+ [3]qxyyxyxxyxx+ [3]qxyxyyxxyxx+ [3]qxyyxxyxyxx, [3]qxyyxxyyxxx+ xyxyxxyyxx+ [3]qxyyxxxyyxx+ xyyxxyxyxx 5 5 xyxyxyxyxy, xyxyxyxyyx+ xyxyxyyxyx+ xyxyyxyxyx+ xyyxyxyxyx, xyxyxyyxxy + xyxyyxyxxy + xyyxyxyxxy + xyxyyxxyxy +xyyxxyxyxy + xyyxyxxyxy, xyyxxyyxyx+ xyyxxyxyyx+ xyyxyxxyyx+ xyxyyxxyyx, xyyxxyyxxy, [3]qxyyxxyxyyx+ xyxyxxyyxy + [3]qxyyxxxyyxy + [3] 2 qxyyxxxyyyx +[3]qxyxyxxyyyx+ xyxyxyxyyx+ xyyxxyxyxy, xyxyyxxyxy + [3]qxyxyyxxxyy + 2xyxyxyxxyy + xyyxyxxyxy +[3]qxyyxyxxxyy + [2] 2 qxyyxxyxxyy + 2xyyxxyxyxy + [3]qxyyxxxyxyy +[3]qxyyxxxyyxy + xyxyxxyxyy + xyxyxxyyxy 4 6 [2]q[3]qxyyxyxxyyy + [2]q[3]qxyxyyxxyyy + [2]q[3]qxyyxxyxyyy +[2]q[3]qxyyxxyyxyy + [2]qxyxyxyyxyy + [2]qxyxyyxyxyy +[2]qxyyxyxyxyy + [2]q[3]qxyxyxyxyyy + [3]q[4]qxyxyxxyyyy +[3]2q[4]qxyyxxxyyyy + [3]q[4]qxyyxxyxyyy 16 Appendix D: Some matrix representations In this appendix we consider the Uq(ŝl2)-module U from Definition 10.3. We display the matrices that represent the actions of E0, F0,K0, E1, F1,K1, D on the bases in Ap- pendix C. On U(0, 0): K0 : ( q ) , K1 : ( 1 ) , D : ( 1 ) . From U(1, 0) to U(0, 0): E0 : ( 1 ) , E1 : ( 0 ) From U(0, 0) to U(1, 0): F0 : ( 1 ) , F1 : ( 0 ) On U(1, 0): K0 : ( q−1 ) , K1 : ( q2 ) , D : ( q−1 ) . From U(1, 1) to U(1, 0): E0 : ( 0 ) , E1 : ( 1 ) From U(1, 0) to U(1, 1): F0 : ( 0 ) , F1 : ( [2]q ) 674 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 On U(1, 1): K0 : ( q ) , K1 : ( 1 ) , D : ( q−1 ) . From U(2, 1) +U(1, 2) to U(1, 1): E0 : ( 1 0 ) , E1 : ( 0 1 ) From U(1, 1) to U(2, 1) +U(1, 2): F0 : ( 1 0 ) , F1 : ( 0 [2]q ) On U(2, 1) +U(1, 2): K0 : diag(q −1, q3), K1 : diag(q 2, q−2), D : diag(q−2, q−1). From U(2, 2) to U(2, 1) +U(1, 2): E0 : ( 0 0 0 1 ) , E1 : ( 1 0 0 0 ) From U(2, 1) +U(1, 2) to U(2, 2): F0 : ( 0 1 0 [3]q ) , F1 : ( [2]q 0 [2]q 0 ) On U(2, 2): K0 : diag(q, q), K1 : diag(1, 1), D : diag(q −2, q−2). From U(3, 2) +U(2, 3) to U(2, 2): E0 : ( 1 0 0 0 1 0 ) , E1 : ( 0 0 1 0 0 1 ) From U(2, 2) to U(3, 2) +U(2, 3): F0 : 1 10 [2]2q 0 0  , F1 :  0 00 0 [2]q 0  On U(3, 2) +U(2, 3): K0 : diag(q −1, q−1, q3), K1 : diag(q 2, q2, q−2), D : diag(q−3, q−3, q−2). From U(4, 2) +U(3, 3) to U(3, 2) +U(2, 3): E0 :  1 0 0 0[3]q 0 0 0 0 0 0 1  , E1 : 0 1 0 00 0 1 0 0 0 0 0  P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 675 From U(3, 2) +U(2, 3) to U(4, 2) +U(3, 3): F0 :  0 1 0 0 0 2 0 0 [2]2q 0 0 [3]q  , F1 :  0 0 0 [2]q 0 0 0 [2]q 0 [2]q 0 0  On U(4, 2) +U(3, 3): K0 : diag(q −3, q, q, q), K1 : diag(q 4, 1, 1, 1), D : diag(q−4, q−3, q−3, q−3). From U(4, 3) +U(3, 4) to U(4, 2) +U(3, 3): E0 :  0 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 1 0 0 0  , E1 :  0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1  From U(4, 2) +U(3, 3) to U(4, 3) +U(3, 4): F0 :  0 1 0 2 0 0 0 [2]2q 0 0 1 0 0 0 0 0 0 0 0 0  , F1 :  [2]q 0 0 0 [2]q 0 0 0 [4]q 0 0 0 0 0 [2]q 0 0 [2]q 0 0  On U(4, 3) +U(3, 4): K0 : diag(q −1, q−1, q−1, q3, q3), K1 : diag(q 2, q2, q2, q−2, q−2), D : diag(q−4, q−4, q−4, q−3, q−3). From U(5, 3) +U(4, 4) to U(4, 3) +U(3, 4): E0 :  2 0 0 0 0 0 [3]q 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1  , E1 :  0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0  From U(4, 3) +U(3, 4) to U(5, 3) +U(4, 4): F0 :  0 1 0 0 0 0 0 0 0 3 0 0 0 [3]q 0 0 0 0 0 [2]2q 0 0 0 1 0 0 0 0 0 [3]q  , F1 :  0 0 0 0 0 [2]q 0 [2]q 0 0 0 [2]q [2]q 0 0 0 [2]q [2]q 0 0 0 0 [2]q[3]q 0 0 [2]q 0 0 0 0  On U(5, 3) +U(4, 4): K0 : diag(q −3, q, q, q, q, q), K1 : diag(q 4, 1, 1, 1, 1, 1), D : diag(q−5, q−4, q−4, q−4, q−4, q−4). 676 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 From U(5, 4) +U(4, 5) to U(5, 3) +U(4, 4): E0 :  0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0  , E1 :  1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 [3]q 0 0 0 0 0 0 1 0  From U(5, 3) +U(4, 4) to U(5, 4) +U(4, 5): F0 :  0 0 0 1 0 0 0 1 0 0 0 3 0 0 [2]2q 0 0 0 0 0 1 0 1 0 0 0 0 0 0 [2]2q 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  , F1 :  [4]q 0 0 0 0 0 3[2]q 0 0 0 0 0 [2]3q 0 0 0 0 0 [2]q[3]q 0 0 0 0 0 2[2]q 0 0 0 0 0 0 0 0 [2]q [2]q 0 0 [2]q 0 0 0 0 0 0 0 0 [2]q 0  On U(5, 4) +U(4, 5): K0 : diag(q −1, q−1, q−1, q−1, q−1, q3, q3, q3),K1 : diag(q 2, q2, q2, q2, q2, q−2, q−2, q−2), D : diag(q−5, q−5, q−5, q−5, q−5, q−4, q−4, q−4). From U(6, 4) +U(5, 5) +U(4, 6) to U(5, 4) +U(4, 5): E0 :  1 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 [3]q 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 [3]q 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0  , E1 :  0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [2]q[3]q 0 0 0 0 0 0 0 0 0 [2]q 0 0 0 0 0 0 0 0 0 [4]q  P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 677 From U(5, 4) +U(4, 5) to U(6, 4) +U(5, 5) +U(4, 6): F0 :  0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 [3]q 0 0 0 0 0 0 0 [2]2q 0 0 0 0 0 0 [3]q 0 0 0 0 0 0 0 [2]2q 0 0 0 0 0 0 0 0 0 [3]q 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0  , F1 :  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3[2]q [2]q 0 0 0 0 0 0 0 [2]q 0 0 0 0 0 0 2[2]q 0 0 0 [2]q 0 0 0 [2]q 0 0 [2]q [2]q 0 0 0 [2]3q 0 [2]q 0 0 0 0 0 0 0 0 [2]q 0 0 0 0 [2]q[3]q 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1  On U(6, 4) +U(5, 5) +U(4, 6): K0 : diag(q −3, q−3, q, q, q, q, q, q, q, q5), K1 : diag(q 4, q4, 1, 1, 1, 1, 1, 1, 1, q−4), D : diag(q−6, q−6, q−5, q−5, q−5, q−5, q−5, q−5, q−5, q−4). 17 Appendix E: Some linear algebra In this appendix we consider the following situation. Let V denote an infinite-dimensional vector space. Let S : V → V and T : V → V denote F-linear maps. Assume that S is locally nilpotent and ST − q2TS = I. (17.1) We will show that S is surjective and T is injective. We remark that the surjectivity of S is used in the proof of Lemma 11.10, and the injectivity of T is used to obtain the surjectivity of S. Lemma 17.1. The map T is injective. Proof. Let v ∈ V such that Tv = 0. We show that v = 0. For n ≥ 1, use (17.1) and induction on n to obtain TSnv = −q−n−1[n]qSn−1v. (17.2) Since S is locally nilpotent, there exists n ≥ 1 such that Snv = 0. By applying (17.2) repeatedly, we see that each of Snv, Sn−1v, . . . Sv, v is equal to 0. In particular v = 0. 678 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 For n ∈ N, we adjust (17.1) to obtain ST − qn[n+ 1]qI = q2 ( TS − qn−1[n]qI ) . (17.3) Therefore, the kernel of ST − qn[n + 1]qI is equal to the kernel of TS − qn−1[n]qI . Let Vn denote this common kernel. By construction( ST − qn[n+ 1]qI ) Vn = 0, ( TS − qn−1[n]qI ) Vn = 0. (17.4) Note that the sum ∑ n∈N Vn is direct. For notational convenience define V−1 = 0. Lemma 17.2. We have ker(S) = V0. Proof. By the discussion below (17.3), we obtain ker(TS) = V0. The map T is injective by Lemma 17.1, so ker(S) = ker(TS). Therefore ker(S) = V0. Lemma 17.3. For n ∈ N we have SVn ⊆ Vn−1, TVn ⊆ Vn+1. Proof. First we verify SVn ⊆ Vn−1. For n = 0 this holds by Lemma 17.2. For n ≥ 1 we use (17.4) to obtain( ST − qn−1[n]qI ) SVn = S ( TS − qn−1[n]qI ) Vn = S0 = 0, so SVn ⊆ Vn−1. Next we verify TVn ⊆ Vn+1. For n ≥ 0 we have( TS − qn[n+ 1]qI ) TVn = T ( ST − qn[n+ 1]qI ) Wn = T0 = 0, so TVn ⊆ Vn+1. Lemma 17.4. For n ≥ 1 the following maps are inverses: S : Vn → Vn−1, q1−n[n]−1q T : Vn−1 → Vn. (17.5) Proof. By (17.4) we have (STn − I)Vn−1 = 0 and (TnS − I)Vn = 0, where Tn = q1−n[n]−1q T . Lemma 17.5. For n ≥ 1 the maps S : Vn → Vn−1, T : Vn−1 → Vn are bijections. Proof. By Lemma 17.4. Lemma 17.6. For n ∈ N, ker(Sn+1) = V0 + V1 + · · ·+ Vn. (17.6) Proof. We use induction on n. First assume that n = 0. Then (17.6) holds by Lemma 17.2. Next assume that n ≥ 1. The inclusion ⊇ in (17.6) holds by Lemma 17.3. We next obtain the inclusion ⊆ in (17.6). Let v ∈ ker(Sn+1). We will show that v ∈ V0 + V1 + · · ·+ Vn. We have 0 = Sn+1v = SnSv, so by induction Sv ∈ V0+V1+· · ·+Vn−1. By Lemma 17.5, there exists w ∈ V1 + V2 + · · · + Vn such that Sw = Sv. Therefore S(w − v) = 0, so w − v ∈ V0. By these comments v ∈ V0 + V1 + · · ·+ Vn. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 679 Lemma 17.7. We have V = ∑ n∈N Vn. Proof. Since S is locally nilpotent, we have V = ∪n∈Nker(Sn+1). The result follows from this and Lemma 17.6. Lemma 17.8. The map S is surjective. Proof. By Lemma 17.7 and since Vn = SVn+1 for n ∈ N. ORCID iDs Paul Terwilliger https://orcid.org/0000-0003-0942-5489 References [1] H. Alnajjar and B. Curtin, A family of tridiagonal pairs, Linear Algebra Appl. 390 (2004), 369–384, doi:10.1016/j.laa.2004.05.003, https://doi.org/10.1016/j.laa.2004. 05.003. [2] H. Alnajjar and B. Curtin, A family of tridiagonal pairs related to the quantum affine algebra Uq(ŝl2), Electron. J. Linear Algebra 13 (2005), 1–9, doi:10.13001/1081-3810.1147, https: //doi.org/10.13001/1081-3810.1147. [3] H. Alnajjar and B. Curtin, A bilinear form for tridiagonal pairs of q-Serre type, Linear Alge- bra Appl. 428 (2008), 2688–2698, doi:10.1016/j.laa.2007.12.015, https://doi.org/10. 1016/j.laa.2007.12.015. [4] S. Ariki, Representations of Quantum Algebras and Combinatorics of Young Tableaux, vol- ume 26 of University Lecture Series, American Mathematical Society, Providence, RI, 2002. [5] P. Baseilhac, The alternating presentation of Uq(ĝl2) from Freidel-Maillet algebras, Nuclear Phys. B 967 (2021), Paper No. 115400, 48, doi:10.1016/j.nuclphysb.2021.115400, https: //doi.org/10.1016/j.nuclphysb.2021.115400. [6] P. Baseilhac, On the second realization for the positive part of Uq(ŝl2) of equitable type, 2022, arXiv:2106.11706 [math.CO]. [7] P. Baseilhac and S. Belliard, The half-infinite XXZ chain in Onsager’s approach, Nuclear Phys. B 873 (2013), 550–584, doi:10.1016/j.nuclphysb.2013.05.003, https://doi.org/ 10.1016/j.nuclphysb.2013.05.003. [8] P. Baseilhac and K. Koizumi, A new (in)finite-dimensional algebra for quantum integrable models, Nuclear Phys. B 720 (2005), 325–347, doi:10.1016/j.nuclphysb.2005.05.021, https: //doi.org/10.1016/j.nuclphysb.2005.05.021. [9] P. Baseilhac and T. Kojima, Correlation functions of the half-infinite XXZ spin chain with a triangular boundary, Nucl. Phys., B 880 (2014), 378–413, doi:10.1016/j.nuclphysb.2014.01. 011, https://doi.org/10.1016/j.nuclphysb.2014.01.011. [10] P. Baseilhac and K. Shigechi, A new current algebra and the reflection equation, Lett. Math. Phys. 92 (2010), 47–65, doi:10.1007/s11005-010-0380-x, https://doi.org/10.1007/ s11005-010-0380-x. [11] J. Beck, Braid group action and quantum affine algebras, Commun. Math. Phys. 165 (1994), 555–568, doi:10.1007/bf02099423, https://doi.org/10.1007/bf02099423. [12] J. Beck, V. Chari and A. Pressley, An algebraic characterization of the affine canonical basis, Duke Math. J. 99 (1999), 455–487, doi:10.1215/S0012-7094-99-09915-5, https://doi. org/10.1215/S0012-7094-99-09915-5. 680 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 [13] R. A. Brualdi, Introductory Combinatorics, Pearson Prentice Hall, Upper Saddle River, NJ, 2010. [14] R. W. Carter, Lie Algebras of Finite and Affine Type, Cambridge Studies in Advanced Math- ematics, Cambridge University Press, Cambridge, 2005, doi:10.1017/cbo9780511614910, https://doi.org/10.1017/cbo9780511614910. [15] V. Chari and A. Pressley, Quantum affine algebras, Comm. Math. Phys. 142 (1991), 261–283, http://projecteuclid.org/euclid.cmp/1104248585. [16] I. Damiani, A basis of type Poincaré-Birkhoff-Witt for the quantum algebra of ŝl(2), J. Algebra 161 (1993), 291–310, doi:10.1006/jabr.1993.1220, https://doi.org/10.1006/jabr. 1993.1220. [17] B. Davies, O. Foda, M. Jimbo, T. Miwa and A. Nakayashiki, Diagonalization of the XXZ Hamiltonian by vertex operators, Commun. Math. Phys. 151 (1993), 89–153, doi:10.1007/ bf02096750, https://doi.org/10.1007/bf02096750. [18] J. A. Green, Shuffle algebras, Lie algebras and quantum groups, volume 9 of Textos Mat., Sér. B, Universidade de Coimbra, Departamento de Matemática, Coimbra, 1995. [19] P. Grossé, On quantum shuffle and quantum affine algebras, J. Algebra 318 (2007), 495–519, doi:10.1016/S0021-8693(03)00307-7, https://doi.org/10.1016/ S0021-8693(03)00307-7. [20] J. Hong and S.-J. Kang, Introduction to Quantum Groups and Crystal Bases, volume 42 of Grad. Stud. Math., American Mathematical Society (AMS), Providence, RI, 2002. [21] T. Ito, K. Tanabe and P. Terwilliger, Some algebra related to P - and Q-polynomial association schemes, in: Codes and Association Schemes (Piscataway, NJ, 1999), Amer. Math. Soc., Prov- idence, RI, volume 56 of DIMACS Ser. Discrete Math. Theoret. Comput. Sci., pp. 167–192, 2001, doi:10.1090/dimacs/056/14, https://doi.org/10.1090/dimacs/056/14. [22] T. Ito and P. Terwilliger, The shape of a tridiagonal pair., J. Pure Appl. Algebra 188 (2004), 145–160, doi:10.1016/j.jpaa.2003.10.002, https://doi.org/10.1016/j. jpaa.2003.10.002. [23] T. Ito and P. Terwilliger, The q-tetrahedron algebra and its finite dimensional irreducible mod- ules, Commun. Algebra 35 (2007), 3415–3439, doi:10.1080/00927870701509180, https: //doi.org/10.1080/00927870701509180. [24] T. Ito and P. Terwilliger, Tridiagonal pairs and the quantum affine algebra Uq(ŝl2)., Ramanu- jan J. 13 (2007), 39–62, doi:10.1007/s11139-006-0242-4, https://doi.org/10.1007/ s11139-006-0242-4. [25] T. Ito and P. Terwilliger, Two non-nilpotent linear transformations that satisfy the cubic q-Serre relations, J. Algebra Appl. 6 (2007), 477–503, doi:10.1142/s021949880700234x, https:// doi.org/10.1142/s021949880700234x. [26] T. Ito and P. Terwilliger, Distance-regular graphs and the q-tetrahedron algebra, Eur. J. Comb. 30 (2009), 682–697, doi:10.1016/j.ejc.2008.07.011, https://doi.org/10.1016/j. ejc.2008.07.011. [27] M. Jimbo and T. Miwa, Algebraic Analysis of Solvable Lattice Models, volume 85 of CBMS Regional Conference Series in Mathematics, Published for the Conference Board of the Math- ematical Sciences, Washington, DC; by the American Mathematical Society, Providence, RI, 1995. [28] V. G. Kac, Infinite Dimensional Lie Algebras, Cambridge University Press, Cambridge, 3rd edition, 1990. P. Terwilliger : Using a q-shuffle algebra to describe the basic module V (Λ0) . . . 681 [29] B. Leclerc, Dual canonical bases, quantum shuffles and q-characters, Math. Z. 246 (2004), 691–732, doi:10.1007/s00209-003-0609-9, https://doi.org/10.1007/ s00209-003-0609-9. [30] G. Lusztig, Introduction to Quantum Groups, volume 110 of Progress in Mathematics, Birkhäuser Boston, Inc., Boston, MA, 1993, doi:10.1007/978-0-8176-4717-9, https:// doi.org/10.1007/978-0-8176-4717-9. [31] A. Negut and A. Tsymbaliuk, Quantum loop groups and shuffle algebras via lyndon words, 2021, arXiv:2102.11269 [math.CO]. [32] S. Post and P. Terwilliger, An infinite-dimensional □q-module obtained from the q-shuffle alge- bra for affine sl2, SIGMA, Symmetry Integrability Geom. Methods Appl. 16 (2020), paper 037, 35, doi:10.3842/SIGMA.2020.037, https://doi.org/10.3842/SIGMA.2020.037. [33] M. Rosso, Groupes quantiques et algèbres de battage quantiques, C. R. Acad. Sci. Paris 320 (1995), 145–148. [34] M. Rosso, Quantum groups and quantum shuffles, Invent. Math. 133 (1998), 399–416, doi: 10.1007/s002220050249, https://doi.org/10.1007/s002220050249. [35] R. P. Stanley, Enumerative Combinatorics. Vol. 2., volume 62 of Cambridge Studies in Ad- vanced Mathematics, Cambridge University Press, Cambridge, 1999. [36] P. Terwilliger, Two relations that generalize the q-Serre relations and the Dolan-Grady rela- tions., in: Physics and Combinatorics. Proceedings of the international workshop, Nagoya, Japan, August 23–27, 1999, World Scientific, Singapore, pp. 377–398, 2001, doi:10.1142/ 9789812810199 0013, https://doi.org/10.1142/9789812810199_0013. [37] P. Terwilliger, The q-Onsager algebra and the positive part of Uq(ŝl2), Linear Algebra Appl. 521 (2017), 19–56, doi:10.1016/j.laa.2017.01.027, https://doi.org/10.1016/ j.laa.2017.01.027. [38] P. Terwilliger, The alternating central extension for the positive part of Uq(ŝl2), Nuclear Phys. B 947 (2019), 114729, 25, doi:10.1016/j.nuclphysb.2019.114729, https://doi.org/10. 1016/j.nuclphysb.2019.114729. [39] P. Terwilliger, The alternating PBW basis for the positive part of Uq(ŝl2), J. Math. Phys. 60 (2019), 071704, 27, doi:10.1063/1.5091801, https://doi.org/10.1063/1. 5091801. [40] P. Terwilliger, Using Catalan words and a q-shuffle algebra to describe a PBW basis for the positive part of Uq(ŝl2), J. Algebra 525 (2019), 359–373, doi:10.1016/j.jalgebra.2019.02.010, https://doi.org/10.1016/j.jalgebra.2019.02.010. [41] P. Terwilliger, The algebra U+q and its alternating central extension U+q , 2021, arXiv:2106.14884 [math.CO]. [42] P. Terwilliger, The alternating central extension of the q-Onsager algebra, Commun. Math. Phys. 387 (2021), 1771–1819, doi:10.1007/s00220-021-04171-2, https://doi.org/10. 1007/s00220-021-04171-2. [43] P. Terwilliger, The compact presentation for the alternating central extension of the q-Onsager algebra, 2021, arXiv:2103.11229 [math.CO]. [44] P. Terwilliger, The q-Onsager algebra and its alternating central extension, Nucl. Phys., B 975 (2022), 37, doi:10.1016/j.nuclphysb.2022.115662, id/No 115662, https://doi.org/10. 1016/j.nuclphysb.2022.115662. [45] P. Terwilliger, Using Catalan words and a q-shuffle algebra to describe the Beck PBW basis for the positive part of Uq(ŝl2), J. Algebra 604 (2022), 162–184, doi:10.1016/j.jalgebra.2022.04. 013, https://doi.org/10.1016/j.jalgebra.2022.04.013. 682 Ars Math. Contemp. 23 (2023) #P4.10 / 649–682 [46] P. M. Terwilliger, A compact presentation for the alternating central extension of the positive part of Uq(ŝl2), Ars Math. Contemp. 22 (2022), 24, doi:10.26493/1855-3974.2669.58c, id/No 1, https://doi.org/10.26493/1855-3974.2669.58c. [47] J. Xiao, H. Xu and M. Zhao, On bases of quantum affine algebras, 2021, arXiv:2107.08631 [math.CO]. [48] Y. Yang, Finite-dimensional irreducible □q-modules and their Drinfel’d polynomials, Linear Algebra Appl. 537 (2018), 160–190, doi:10.1016/j.laa.2017.10.002, https://doi.org/ 10.1016/j.laa.2017.10.002. Author Guidelines Before submission Papers should be written in English, prepared in LATEX, and must be submitted as a PDF file. The title page of the submissions must contain: • Title. The title must be concise and informative. • Author names and affiliations. For each author add his/her affiliation which should include the full postal address and the country name. If avilable, specify the e-mail address of each author. Clearly indicate who is the corresponding author of the paper. • Abstract. A concise abstract is required. The abstract should state the problem stud- ied and the principal results proven. • Keywords. Please specify 2 to 6 keywords separated by commas. • Mathematics Subject Classification. Include one or more Math. Subj. Class. (2020) codes – see https://mathscinet.ams.org/mathscinet/msc/msc2020.html. After acceptance Articles which are accepted for publication must be prepared in LATEX using class file amcjoucc.cls and the bst file amcjoucc.bst (if you use BibTEX). If you don’t use BibTEX, please make sure that all your references are carefully formatted following the examples provided in the sample file. All files can be found on-line at: https://amc-journal.eu/index.php/amc/about/submissions/#authorGuidelines Abstracts: Be concise. As much as possible, please use plain text in your abstract and avoid complicated formulas. Do not include citations in your abstract. All abstracts will be posted on the website in fairly basic HTML, and HTML can’t handle complicated formulas. It can barely handle subscripts and greek letters. Cross-referencing: All numbering of theorems, sections, figures etc. that are referenced later in the paper should be generated using standard LATEX \label{. . . } and \ref{. . . } commands. See the sample file for examples. Theorems and proofs: The class file has pre-defined environments for theorem-like state- ments; please use them rather than coding your own. Please use the standard \begin{proof} . . . \end{proof} environment for your proofs. Spacing and page formatting: Please do not modify the page formatting and do not use \medbreak, \bigbreak, \pagebreak etc. commands to force spacing. In general, please let LATEX do all of the space formatting via the class file. The layout editors will modify the formatting and spacing as needed for publication. Figures: Any illustrations included in the paper must be provided in PDF format, or via LATEX packages which produce embedded graphics, such as TikZ, that compile with PdfLATEX. (Note, however, that PSTricks is problematic.) Make sure that you use uniform lettering and sizing of the text. If you use other methods to generate your graphics, please provide .pdf versions of the images (or negotiate with the layout editor assigned to your article). xxxiii Subscription Yearly subscription: 150 EUR Any author or editor that subscribes to the printed edition will receive a complimentary copy of Ars Mathematica Contemporanea. Subscription Order Form Name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-mail: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Postal Address: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I would like to subscribe to receive . . . . . . copies of each issue of Ars Mathematica Contemporanea in the year 2023. I want to renew the order for each subsequent year if not cancelled by e-mail: □ Yes □ No Signature: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Please send the order by mail, by fax or by e-mail. By mail: Ars Mathematica Contemporanea UP FAMNIT Glagoljaška 8 SI-6000 Koper Slovenia By fax: +386 5 611 75 71 By e-mail: info@famnit.upr.si xxxiv Printed in Slovenia by Tiskarna Koštomaj d.o.o.