Harmonic Analysis in One-Parameter Metabelian Nilmanifolds

Let $G$ be a connected, simply connected one-parameter metabelian nilpotent Lie group, that means, the corresponding Lie algebra has a one-codimensional abelian subalgebra. In this article we show that $G$ contains a discrete cocompact subgroup. Given a discrete cocompact subgroup $\Gamma$ of $G$, we define the quasi-regular representation $\tau = {\rm ind}_\Gamma^G 1$ of $G$. The basic problem considered in this paper concerns the decomposition of $\tau$ into irreducibles. We give an orbital description of the spectrum, the multiplicity function and we construct an explicit intertwining operator between $\tau$ and its desintegration without considering multiplicities. Finally, unlike the Moore inductive algorithm for multiplicities on nilmanifolds, we carry out here a direct computation to get the multiplicity formula.

It is known [8, p. 23] that the representation τ splits into a discrete direct sum of a countable number of irreducible unitary representations, each π with finite multiplicity m(π). We write For any given nilpotent Lie group G with discrete cocompact subgroup Γ, there are two general problems to consider. The first is to determine the spectrum (G : Γ) and the multiplicity function m(π) of these representations. The second is to construct an explicit intertwining operator between τ and its decomposition into irreducibles. A necessary and sufficient condition for π to occur in τ was obtained, by Calvin C. Moore [14], in the special case in which Γ is a lattice subgroup of G (i.e., Γ is a discrete cocompact subgroup of G and log(Γ) is an additive subgroup of the Lie algebra g of G). Moreover, Moore determined an algorithm which expresses m(π) in terms of multiplicities of certain representations of a subgroup of codimension one in G. Later, R. Howe and L. Richardson [11,18] independently obtained a closed formula for the multiplicities for general Γ. In [15], C. Moore and J. Wolf determine explicitly which square integrable representations of G occur in the decomposition (1) and give a method for calculating the multiplicities. Using the Poisson summation and Selberg trace formulas, L. Corwin and F. Greenleaf gave a formula for m(π) that depended only on the coadjoint orbit in g * corresponding to π via Kirillov theory and the structure of Γ [6] (see also [3,4,5,6,7]).
Our main goals in this paper are twofold. The first is to give an orbital description of the decomposition of τ into irreducibles, in the case when G is a connected, simply connected one-parameter metabelian nilpotent Lie group. Such a decomposition has two components, the spectrum (G : Γ) and the multiplicity function π → m(π). We give orbital description of both. The second main goal is to give an explicit intertwining operator between τ and its desintegration.
This paper is organized as follows. In Section 2, we establish notations and recall a few standard facts about representation theory, rational structure and cocompact subgroups of a connected simply connected nilpotent Lie groups. Section 3 is devoted to present some results which will be used in the next sections. In Section 4 we prove, first, that a one-parameter metabelian nilpotent Lie group admits a discrete uniform subgroup Γ (i.e., the homogeneous space G/Γ is compact). Next, for a fixed discrete uniform subgroup Γ we prove the existence of a one-codimensional abelian rational ideal M (i.e., M ∩ Γ is a discrete uniform subgroup of M ). Furthermore, we give a necessary and sufficient condition for the uniqueness of M . In Section 5, we pick from a strong Malcev basis strongly based on Γ, an orbital description of the spectrum (G : Γ). We obtain the following decomposition where Σ is a crosssection for Γ-orbits in a certain subspace W ⊂ g * and V ⊂ g * (more details in Theorem 4). We describe an intertwining operator U of τ and ρ, defined for all ξ ∈ C(G/Γ) and g ∈ G by This operator does not take into account the multiplicities of the decomposition of τ . As a consequence, we give an orbital description of the multiplicity function.

Notations and basic facts
The purpose of this section is to establish notations which will be used in the sequel and recall some basic definitions and results which we shall freely use afterwards without mentioning them explicitly. Let G be a connected and simply connected nilpotent Lie group with Lie algebra g, then the exponential map exp : g → G is a diffeomorphism. Let log : G → g denote the inverse of exp.

Induced representation
Starting from a closed subgroup H of a nilpotent Lie group G and a unitary representation σ of H in a Hilbert space H σ , let us construct a unitary representation of G. We realize the unitary representation Ind G H σ of G, the representation induced on G from the representation σ of H, by left translations on the completion of the space of continuous functions F on G with values in H σ satisfying and having a compact support modulo H, provided with the norm

The orbit theory
Suppose G is a nilpotent Lie group with Lie algebra g. G acts on g (respectively g * ) by the adjoint (respectively co-adjoint) action. For l ∈ g * , let be the stabilizer of l in g which is actually the Lie algebra of the Lie subgroup So, it is clear that g(l) is the radical of the skew-symmetric bilinear form B l on g defined by A subspace b(l) of the Lie algebra g is called a polarization for l ∈ g * if it is a maximal dimensional isotropic subalgebra with respect to B l . We have the following equality dim(b(l)) = 1 2 dim(g) + dim(g(l)) .
If b(l) is a polarization for l ∈ g * let B(l) = exp(b(l)) be the connected subgroup of G with Lie algebra b(l) and define a character of B(l) by the formula χ l (exp(X)) = e 2iπ l,X , ∀ X ∈ b(l). Now, we recall the Kirillov orbital parameters. We denote byĜ the unitary dual of G, i.e. the set of all equivalence classes of irreducible unitary representations of G. We shall sometimes identify the equivalence class [π] with its representative π and we denote the equivalence relation between two representations π 1 and π 2 by π 1 ≃ π 2 or even by π 1 = π 2 . The dual spaceĜ of G is parameterized canonically by the orbital space g * /G. More precisely, for l ∈ g * we may find a real polarization b(l) for l. Then the representation π l = Ind G B(l) χ l is irreducible; its class is independent of the choice of b(l); the Kirillov mapping l → π l is surjective and factors to a bijection g * /G →Ĝ. Given π ∈Ĝ, we write Ω(π) ∈ g * /G to denote the inverse image of π under the Kirillov mapping Kir G .

Rational structures and uniform subgroups
In this section we present some results on discrete uniform subgroups of nilpotent Lie groups.
Let G be a nilpotent, connected and simply connected real Lie group and let g be its Lie algebra. We say that g (or G) has a rational structure if there is a Lie algebra g Q over Q such that g ∼ = g Q ⊗ R. It is clear that g has a rational structure if and only if g has an R-basis (X 1 , . . . , X n ) with rational structure constants.
A discrete subgroup Γ is called uniform in G if the quotient space G/Γ is compact. The homogeneous space G/Γ is called a compact nilmanifold. A proof of the next result can be found in Theorem 7 of [12] or in Theorem 2.12 of [17].
Theorem 1 (the Malcev rationality criterion). Let G be a simply connected nilpotent Lie group, and let g be its Lie algebra. Then G admits a uniform subgroup Γ if and only if g admits a basis (X 1 , . . . , X n ) such that where the constants c ijk are all rational.
More precisely, if G has a uniform subgroup Γ, then g has a rational structure such that Conversely, if g has a rational structure given by some Q-algebra g Q ⊂ g, then G has a uniform subgroup Γ such that log(Γ) ⊂ g Q (see [2] or [12]).

Definition 1 ([2]
). Let g be a nilpotent Lie algebra and let B = (X 1 , . . . , X n ) be a basis of g.
We say that B is a strong Malcev basis for Let Γ be a discrete uniform subgroup of G. A strong Malcev basis (X 1 , . . . , X n ) for g is said to be strongly based on Γ if Such a basis always exists (see [2,13]).

Rational subgroups
Definition 2 (rational subgroup). Let G be a connected simply connected nilpotent Lie group with Lie algebra g. We suppose that g has rational structure given by g Q .
(2) A connected, closed subgroup H of G is called rational if its Lie algebra h is rational. Definition 3 (subgroup with good Γ-heredity, [16]). Let Γ be a discrete uniform subgroup in a locally compact group G and H a closed subgroup in G. We say that H is a subgroup with good Γ-heredity if the intersection Γ ∩ H is a discrete uniform subgroup of H.
. Let G be a connected, simply connected nilpotent Lie group with Lie algebra g, let Γ be a discrete uniform subgroup of G, and give g the rational structure g Q = Q-span {log(Γ)}. Let H be a Lie subgroup of G. Then the following statements are equivalent (1) H is rational; (2) H is a subgroup with good Γ-heredity; (3) The group H is Γ-closed (i.e., the set HΓ is closed in G).
A proof of the next result can be found in Proposition 5.3.2 of [2]. Proposition 1. Let Γ be discrete uniform subgroup in a nilpotent Lie group G, and let H 1 H 2 · · · H k = G be rational normal subgroups of G. Let h 1 , . . . , h k−1 , h k = g be the corresponding Lie algebras. Then there exists a strong Malcev basis (X 1 , . . . , X n ) for g strongly based on Γ and passing through h 1 , . . . , h k−1 . Now let G be a connected, simply connected nilpotent Lie group with Lie algebra g, and suppose that g has a rational structure given by the discrete uniform subgroup Γ. A real linear . Let G be a nilpotent Lie group with rational structure and let g be its Lie algebra. If l ∈ g * is rational, then its radical g(l) is rational.

Preliminary results
Definition 4. A functional l ∈ g * is said in general position or generic linear functional, if its coadjoint orbit has maximum dimension.
The following proposition will be used in the sequel Proposition 3. Let G = exp(g) be a nilpotent Lie group and Γ a discrete uniform subgroup of G. Then there exist rational generic linear functionals.
Before stating the next result, we introduce some notations and definitions. We first recall the concept of fundamental domain. Let Γ be a discrete uniform subgroup of a connected, simply connected nilpotent Lie group G, and let B = (X 1 , . . . , X n ) be a strong Malcev basis for the Lie algebra g of G strongly based on Γ. Define the mapping E B : R n → G by where T = (t 1 , . . . , t n ) ∈ R n . It is well known that E B is a diffeomorphism [2]. Let and let Then Ω is a fundamental domain for Γ in G [6, Lemma 3.6], and the mapping E B maps the Lebesgue measure dt on I n to the G-invariant probability measure ν on G/Γ, that is, for ϕ in C(G/Γ), we have The following proposition will be used in the sequel.
where X i are the image of X i under the canonical projection p : g → g/m.
Proof . Since M is rational, it follows from Lemma 5.1.4 of [2], that P (Γ) is a uniform subgroup of P (G), where P : G → G/M is the canonical projection. On the other hand, (X k+1 , . . . , X n ) is a strong Malcev basis of g/m strongly based on P (Γ). We conclude by applying (4).
4 On the rational structure of one-parameter metabelian nilpotent Lie groups Definition 6 (one-parameter metabelian nilpotent Lie algebra). A nonabelian nilpotent Lie algebra g is said to be one parameter metabelian nilpotent Lie algebra, if it admits a co-dimensional one abelian ideal in g.
First, we give an important example of one-parameter metabelian nilpotent Lie groups.
Example 1 (the generic filiform nilpotent Lie group). Let G be the generic filiform nilpotent Lie group of dimension n with Lie algebra g, where The Lie brackets given by [X n , X i ] = X i−1 , i = 2, . . . , n − 1, and the nondefined brackets being equal to zero or obtained by antisymmetry. It is clear that g is a one-parameter metabelian nilpotent Lie algebra.
Remark 3. Let g be a one-parameter metabelian nilpotent Lie algebra. Any codimension one abelian ideal m ⊂ g is a common polarization for all functionals l ∈ g * in general position (i.e., l| [g,g] = 0).
Definition 7 (one-parameter metabelian nilmanifold). A factor space of a one-parameter metabelian nilpotent Lie group over a discrete uniform subgroup is called a one-parameter metabelian nilmanifold.
Proposition 5. Let g be a one-parameter metabelian nilpotent Lie algebra. Then we have the following decomposition such that for all i = 1, . . . , p, the subalgebra RX ⊕L n i is the generic filiform nilpotent Lie algebra of dimension n i + 1 and a ⊂ z(g).
Proof . Let I be a one-codimensional abelian ideal of g and let X ∈ g such that Let adX| I be the restriction of adX to I; Remark that ad X acts as a nilpotent linear transformation on I. This completes the proof.

Proposition 6. Let G be a one-parameter metabelian nilpotent Lie group with Lie algebra g.
Then G admits a discrete uniform subgroup.
Proof . This follows at once from the Malcev rationality criterion (Theorem 1) and Proposition 5.
Proposition 7. Let G be a one-parameter metabelian nilpotent Lie group of dimension n. Let Γ be a discrete uniform subgroup of G. Then G admits a rational abelian ideal of codimension one.
Before proving the proposition, we need the following theorem. In this case, we have dim(a) = 1. Moreover, it is clear that By Proposition 5 of [9], the space is rational. Consequently, a is rational subspace in g. Thus a ⊕ g(l) is a rational abelian ideal of g of codimension one.
The next proposition is a generalization of Proposition 3.1 of [10], in which we give a necessary and sufficient condition for uniqueness of the one-codimensional abelian normal subgroup.

Proposition 8. Let G be a one-parameter metabelian nilpotent Lie group. Then G admits a unique one-codimensional abelian normal subgroup if and only if G is not of the form H 3 × R k , where k ∈ N and H 3 is the Heisenberg group of dimension 3.
Proof . The necessity of this condition is evident. We prove the sufficiency. Let m 1 , m 2 be two abelian one-codimensional subalgebras of g. Let l be a linear functional in general position. Since every polarization for l contains g(l) then there exists X ∈ g (see (5)) such that On the other hand, there exist α ∈ R * and u ∈ m 1 such that Since m 1 and m 2 are abelian then Consequently, [X,Y ] is nonzero bracket. Let a be an ideal of g such that g(l) = a⊕R-span {[X,Y ]}.

Construction of intertwining operators
Let G be a one-parameter metabelian nilpotent Lie group of dimension n with Lie algebra g. Let Γ be a discrete uniform subgroup of G. Let M = exp(m), where m is a rational abelian ideal in g of codimension one. Let B = (X 1 , . . . , X n ) be a strong Malcev basis of g strongly based on Γ passing through [g, g] and m. We put [g, g] = R-span{X 1 , . . . , X p } and m = R-span {X 1 , . . . , X n−1 } .
On the other hand, since M is normal in G, we have that Since M is abelian and It follows that l, e ad γ (X i ) ∈ Z. Then exp(−γ) · l ∈ ZX * 1 ⊕ · · · ⊕ ZX * n−1 .
Let C(G/Γ) be the space of all complex valued continuous functions ξ on G satisfying for any g in G and γ in Γ.
This completes the proof of the lemma.
Let Σ be a crosssection for Γ-orbits in W. Let We are now in a position to formulate the following Theorem 4. The operator U defined for all ξ ∈ C(G/Γ) and g ∈ G by is an isometric linear operator having value in the Hilbert space H ρ of ρ and can be extended on L 2 (G/Γ) to an intertwining operator of τ and ρ.
Proof . Clearly for l ∈ Σ ∪ V, the function U(ξ)(l) satisfies the covariance relation (2). First, we establish that U is well defined and isometric. Let ξ ∈ C(G/Γ). Then We proceed to calculate the first sum. For x = (x 1 , . . . , x n−1 ) ∈ R n−1 and t ∈ R, let Next, we compute where we have applied Parseval's equality with respect to the variable t. Summarizing, we have It is clear that W and V 1 are disjoint. In fact, suppose that W ∩ V 1 = ∅. Let l ∈ W ∩ V 1 . The condition l ∈ V 1 implies that l| [g,g] = 0 since [g, g] = RX 1 ⊕ · · · ⊕ RX p (see (6)). This contradicts the fact that l| [g,g] = 0 because l ∈ W. Now, let It is clear that Consequently, we obtain The operator U being now isometric, it can be extended to an isometry of L 2 (G/Γ). It remains to verify that U is an intertwining operator for τ and ρ. It suffices then to prove that U •τ (g)(ξ) = ρ(g) • U(ξ) for every g in G and ξ in C(G/Γ). Let l ∈ Σ, a ∈ G. We compute On the other hand Similarly, we prove that the same equality holds if l ∈ V. Consequently, the following diagram: is commutative.
Next, we show that U is an invertible operator. For this, let where C c (G/M, χ l ) is the space of complex valued continuous functions ξ on G satisfying and having a compact support modulo M .
is well defined.
Then there is an integer n K(l) independent of g ∈ G such that S g is the union of at most n K(l) cosets of Γ ∩ M [3, Lemma 3.2]. This observation shows that the sum over Γ/Γ ∩ M in (13) has at most n K(l) nonzero entries for each g ∈ G. Consequently, the sum (13) is a sum of finite terms.
We define the operator V on the dense subspace H 0 ρ by Proposition 9. The operator V is the inverse of U.
Now, if l ∈ V, the coadjoint orbit Ω(π l ) has only one element and therefore m(π l ) = 1.