Matrix gamma distributions are among the most important matrix-variate laws in multivariate statistical analysis, as they encompass the Wishart distributions - the sample covariance distributions under Gaussianity - and provide a natural model for random covariances in Bayesian multivariate methods. A substantial body of literature explores this class of distributions, traditionally characterized by a shape parameter restricted to the (Gindikin) set {i/2, i ∈ {1, . . . , k − 1}} ∪ ((k − 1)/2, ∞), where k × k is the dimension of the matrix variate. In this paper, we show that matrix-variate gamma distributions can be naturally extended to allow the entire positive half-line as the domain of the shape parameter. This extension not only unifies the well-known singular Wishart and non-singular matrix-variate gamma distributions but also introduces new singular matrix-variate distributions with shape parameters outside the Gindikin set. While permutation invariance is no longer preserved in the singular, non-Wishart case, and its scaling properties require special treatment, our unified framework leads to new representations that bypass the restrictions of the Gindikin set. We provide several elegant and convenient stochastic representations for matrix-variate gamma distributions, which are novel even in the non-singular case. Notably, we demonstrate that the lower triangular matrix in the Cholesky factorization of a gamma-distributed matrix - whether singular or not - follows a triangular matrix-variate Rayleigh distribution, introducing a new class of matrix-valued variables that extends the classical univariate Rayleigh distribution to the matrix domain. We also briefly address statistical issues and potential applications to non-elliptical multivariate heavy tailed data.