A four-directional total variation technique is proposed to encapsulate the spatial contextual information for sparse hyperspectral image (HSI) unmixing. Traditional sparse total variation techniques explore gradient information along with the horizontal and vertical directions. As a result, spatial disparity due to high noise levels within the neighboring pixels are not considered while unmixing. Moreover, oversmoothing due to total variation may depreciate the spatial details in the abundance map. In this context, we propose a four-directional regularization technique (Sparse Unmixing with Splitting Augmented Lagrangian: Four-Directional Total Variation, SUnSAL-4DTV) for sparse unmixing. The four-directional total variation scheme is transformed into the fast-Fourier-transform domain to reduce the higher computational requirements. An alternating-direction-method-of-multipliers-based iterative scheme is proposed for solving the large-scale optimization problem. An adaptive scheme is introduced to update the regularization parameters to ensure faster convergence. Extensive numerical simulations were conducted on both simulated and real hyperspectral datasets to demonstrate the robustness of proposed technique. Comparative analysis on noisy (low signal-to-noise-ratio) HSIs shows the robustness of SUnSAL-4DTV over the state-of-the-art algorithms.