Rasoul Soleimani - Department of Mathematics, Payame Noor University (PNU), ۱۹۳۹۵-۳۶۹۷, Tehran, Iran.

Let F be a free group, \mathcal{V} be a variety of groups defined by the set of laws V\subseteq F and G be a finite \mathcal{V}-nilpotent p-group. The automorphism \alpha of G is said to be a marginal automorphism (with respect to V), if for all x\in G, x^{-۱}x^{\alpha}\in V^{\star}(G), where V^{\star}(G) denotes the marginal subgroup of G. An automorphism \alpha of G is called an IA-automorphism if x^{-۱}x^{\alpha}\in G' for each x\in G. An automorphism \alpha of G is called a class preserving if for all x\in G, there exists an element g_x\in G such that x^{\alpha}=g_x^{-۱}xg_x. Let \operatorname{Aut}^{V^{\star}}(G), \operatorname{Aut}^{G'}(G) and \operatorname{Aut}_c(G) respectively, denote the group of all marginal automorphisms, IA-automorphisms and class preserving automorphisms of G. In this paper, first we give a necessary and sufficient condition on a finite \mathcal{V}-nilpotent p-group G such that each marginal automorphism of G fixes the center of G element-wise. Then we characterize all finite \mathcal{V}-nilpotent p-groups G such that \operatorname{Aut}^{V^{\star}}(G)=\operatorname{Aut}^{G'}(G). Finally, we obtain a necessary and sufficient condition for a finite \mathcal{V}-nilpotent p-group G such that \operatorname{Aut}^{V^{\star}}(G)=\operatorname{Aut}_c(G).

Automorphism group, Marginal automorphism, Variety, Marginal subgroup, Finite p-group