
Journal of Lie Theory 17 (2007), No. 1, 191227 Copyright Heldermann Verlag 2007 Branching of Some Holomorphic Representations of SO(2,n) Henrik Seppänen Dept. of Mathematics, Chalmers University of Technology and Göteborg University, 41296 Göteborg, Sweden henriks@math.chalmers.se [Abstractpdf] We consider the analytic continuation of the weighted Bergman spaces on the Lie ball ${\cal D}=SO(2,n)/S(O(2) \times O(n))$ and the corresponding holomorphic unitary (projective) representations of $SO(2,n)$ on these spaces. These representations are known to be irreducible. Our aim is to decompose them under the subgroup $SO(1,n)$ which acts as the isometry group of a totally real submanifold ${\cal X}$ of ${\cal D}$. We give a proof of a general decomposition theorem for certain unitary representations of semisimple Lie groups. In the particular case we are concerned with, we find an explicit formula for the Plancherel measure of the decomposition as the orthogonalising measure for certain hypergeometric polynomials. Moreover, we construct an explicit generalised Fourier transform that plays the role of the intertwining operator for the decomposition. We prove an inversion formula and a Plancherel formula for this transform. Finally we construct explicit realisations of the discrete part appearing in the decomposition and also for the minimal representation in this family. Keywords: Bounded symmetric domain, Lie group, Lie algebra, unitary representation, spherical function, hypergeometric function, intertwining operator. MSC: 32M15, 22E46, 22E43, 43A90, 32A36 [ Fulltextpdf (302 KB)] for subscribers only. 