We present a novel method for the outdoor scene categorization using 2D convolutional neural networks (CNNs) which take panoramic depth images obtained by a 3D laser scanner as input. We evaluate our approach in two outdoor scene datasets including six categories: coast, forest, indoor parking, outdoor parking, residential area, and urban area. Our results on both datasets (over 94%) outperform previous approaches and show the effectiveness of this approach for outdoor scene categorization using depth images. To analyze our trained networks we visualize the learned features by using two visualization methods.