ARS MATHEMATICA CONTEMPORANEA ISSN 1855-3966 (printed edn.), ISSN 1855-3974 (electronic edn.) ARS MATHEMATICA CONTEMPORANEA 15 (2018) 523-542 https://doi.org/10.26493/1855-3974.1062.ba8 (Also available at http://amc-journal.eu) Wonderful symmetric varieties and Schubert polynomials* Mahir Bilen Can, Michael Joyce Department of Mathematics, Tulane University, New Orleans, LA 70118, USA Benjamin Wyser Department ofMathematics, Oklahoma State University, Stillwater, OK 74078, USA Received 9 June 2016, accepted 25 October 2017, published online 11 September 2018 Extending results of Wyser, we determine formulas for the equivariant cohomology classes of closed orbits of certain families of spherical subgroups of the general linear group on the flag variety. Combining this with a slight extension of results of Can, Joyce and Wyser, we arrive at a family of polynomial identities which show that certain explicit sums of Schubert polynomials factor as products of linear forms. Keywords: Symmetric varieties, Schubert polynomials, wonderful compactification, equivariant co-homology, weak order, parabolic induction. Math. Subj. Class.: 14M27, 05E05, 14M15 1 Introduction Suppose that G is a connected reductive algebraic group over C. Suppose that B D T are a Borel subgroup and a maximal torus of G, respectively, W is the Weyl group, and let t denote the Lie algebra of T. By a classical theorem of Borel [1], the cohomology ring of G/B with rational coefficients is isomorphic to the coinvariant algebra Q[t*]/ZW, where * We thank Michel Brion for many helpful conversations and suggestions. We thank the referee for their careful reading and many helpful suggestions. The first and second author were supported by NSA-AMS Mathematical Sciences Program grant H98230-14-1-142. The second is partially supported by the Louisiana Board of Regents Research and Development Grant 549941C1. The third author was supported by NSF International Research Fellowship 1159045 and hosted by Institut Fourier in Grenoble, France. E-mail address: mcan@tulane.edu (Mahir Bilen Can), mjoyce3@tulane.edu (Michael Joyce), bwyser@okstate.edu (Benjamin Wyser) Abstract ©® This work is licensed under https://creativecommons.org/licenses/by/3.0/ 524 Ars Math. Contemp. 15 (2018) 441-466 IW denotes the ideal generated by homogeneous W-invariant polynomials of positive degree. Any subvariety Y of G/B defines a cohomology class [Y] in H*(G/B). It is then natural to ask for a polynomial in Q[t*] which represents [Y]. In this paper, for certain families of subvarieties of certain G/B, we approach and answer this question in two different ways. Relating the two answers leads in the end to our main result, Theorem 4.1, which, roughly stated, says that certain non-negative linear combinations of Schubert polynomials factor completely into linear forms. Our group of primary interest is G = GLn, with B its Borel subgroup of lower-triangular matrices, and T its maximal torus of diagonal matrices. In this case, there is a canonical basis xi,... ,xn of t* that correspond to the Chern classes of the tautological quotient line bundles on the variety of complete flags G/B. Let Zn denote the center of GLn, consisting of diagonal scalar matrices. Let On denote the orthogonal subgroup of GLn, and let Sp2n denote the symplectic subgroup of GL2n. Denote by GOn (resp. GSp2n) the central extension ZnOn (resp. Z2nSp2n). For any ordered sequence of positive integers m = (pi,... ,Ms) that sum to n, GLn has a Levi subgroup Lm := GLM1 x • • • x GLMs, as well as a parabolic subgroup PM = LM x UM containing B, where UM denotes the unipotent radical of PM. The subgroup Hm := (GOMi x---x GOMs) x UM of GLn is spherical, meaning that it acts on GLn/B with finitely many orbits. Moreover, there is a unique closed HM-orbit YM on GLn/B, which is our object of primary interest. The reason for our interest in this family of orbits is that they correspond to the closed B-orbits on the various G-orbits of the wonderful compactification of the homogeneous space GLn/GOn. This homogeneous space is affine and symmetric, and it is classically known as the space of smooth quadrics in Pn-i. Its wonderful compactification, classically known as the variety of complete quadrics [9, 13], is a G-equivariant projective embedding X which contains it as an open, dense G-orbit, and whose boundary has particularly nice properties. (We recall the definition of the wonderful compactification in Section 2.1.) It turns out that, with minor modifications, our techniques apply also to the wonderful compactification X' of the space GL2n/GSp2n, which parameterizes non-degenerate skew-symmetric bilinear forms on C2n, up to scalar. Letting G = GL2n in this case, the G-orbits on X' are again parametrized by compositions M = (Mi,..., Ms) of n; note that this is of course equivalent to parametrizing them by compositions of 2n with each part being even. Each G-orbit has the form G/H^, with H := (GSp2Mi x---x GSp2Ms) x UM, a spherical subgroup which again acts on GL2n/B with a unique closed orbit Y^. Let us consider two ways in which one might try to compute a polynomial representative of [Ym] (or [Y^]). For the first, note that YM, being an orbit of HM, also admits an action of a maximal torus SM of HM. Thus YM admits a class [YM]S^ in the SM-equivariant cohomology of GLn/B, denoted by HS (GLn/B). In brief, this is a cohomology theory which is sensitive to the geometry of the SM-action on GLn/B. It admits a similar Boreltype presentation, this time as a polynomial ring in two sets of variables (the usual set of x-variables referred to in the second paragraph, along with a second set which consists of M. B. Can et al.: Wonderful symmetric varieties and Schubert polynomials 525 y and z-variables) modulo an ideal. Moreover, the map HS (GLn/B) ^ H*(GLn/B) which sets all of the y and z-variables to 0 sends the equivariant class of any SM-invariant subvariety of GLn/B to its ordinary (non-equivariant) class. Thus if a polynomial representative of [Ym]sm can be computed, one obtains a polynomial representative of [YM] by specializing y, z ^ 0. In [15], this problem is solved for the case in which ^ has only one part, in which case Hm = GOn. Here, we extend the results of [15] to give a formula for the equivariant class [Y„l sm (and [Y' ]s' ) for an arbitrary composition The main general result is Proposition 3.4; it, together with Proposition 3.5, imply the case-specific equivariant formulas given in Corollaries 3.6 and 3.8. The formulas for [YM] and [Y^] obtained from these corollaries (by specializing y and z-variables to 0) are as follows: Corollary 1.1. The ordinary cohomology class of [YM] is represented in H* (G/B) by the formula ... _ _ . Vi=1 / i= 1 Vi+1 3.) So for m = (6,5) as above, we have that gl(x, z) = (xi + x2 - 2zi)(xi + x3 - 2zi)(xi + x4 - 2zi)(xi + x5 - 2zi) (x2 + x3 - 2zi)(x2 + x4 - 2zi), and g2 (x, z) = (xr + xg - 2z2)(xr + xg - 2z2)(xr + xio - 2z2)(xg + xg - 2Z2). Finally, we define a third polynomial hM(x, y, z) in the x, y, and z-variables to simply be hM(x, p(y)), where p denotes restriction from the variables yl,..., yn corresponding to coordinates on the full torus T to the variables yitj, zi on the smaller torus SM. To be more explicit, for each i, j with 1 < i < j < s define hi,j (x, y, z) to be rifc=l nr=jl2(xVi+fc - y^- zj x^+k+y^- zj) if mis even, nr= l (xVi+k - zj) n[=l/2J (xVi+k - yj,i - zj)(xVi+k + yj,i - zj) if Mjis odd. So for the case n = 4, m = (2,2), we have hl,2(x, y, z) = (xi - y2,l - Z2)(xi + y2,l - Z2)(x2 - y2,l - Z2)(x2 + y2,l - Z2), while for the case n = 5, m = (2, 3), we have hl,2(x, y, z) = (xl - Z2)(x2 - Z2)(xl - y2,l - Z2)(xi + y2,l - Z2) (x2 - y2,l - Z2)(x2 + y2,l - Z2). Then we define hM(x, y, z) = JJ hj,j(x, y, z). l (x, y, z) := n P* (x(j), y(j),Zj). i=i We now specialize these formulas to ordinary cohomology (by setting all y and z-variables to 0), in order to prove Corollaries 1.1 and 1.2. First, we define the notations used in those formulas which have not yet been defined. For each i = 1,..., n, let B(p,i) denote the block that the variable xi occurs in, i.e. b(v, i) is the smallest integer j such that j -i. i=\ Then for each i = 1,..., n, define r(v, i) to be R(v,i):= Vj. B(n ,i) i the given x-variable appears in precisely ^j linear forms involving y, z terms associated with block j. The proof of Corollary 1.2 is almost identical, except simpler. □ 4 Factoring sums of Schubert polynomials We end by establishing explicit polynomial identities involving sums of Schubert polynomials, using the cohomological formulae of the preceding section together with the results of [5, 6] which were recalled in Section 2.4. Note that by Brion's formula (1.2) combined with the fact that the Schubert polynomial Sw is a representative of the class of the Schubert variety Xw in H*(G/B), we have the following two families of identities in H*(G/B): ^ 2d(Y-w)6w = 2d(M) J]xf(^i)+5(^i) n ( n (xj + xk)Y (4.1) wew(Ym) i=l i=l \ Vi + l