An improved Nadam optimizer by incorporating gradient clipping for LSTM Network

Ammonia concentration is crucial for evaluating aquaculture water quality, and accurate prediction can facilitate smart aquaculture. Long Short-Term Memory (LSTM) networks perform well in time series prediction, such as ammonia levels, but their effectiveness depends on proper optimization. Nesterov-accelerated Adaptive Moment Estimation (Nadam) combines Nesterov-accelerated gradients with adaptive learning rates to achieve rapid convergence, but it can encounter instability issues like gradient spikes. This study introduces NadamClip, an improved optimizer with two key enhancements: i) Adding a gradient trimming mechanism to reduce gradient spikes by limiting the impact of abnormal gradients during weight updates. ii) Manually adjusting the cropping threshold and testing to find the best gradient threshold to improve training stability and prediction accuracy. Compared with various stochastic and fused gradient descent algorithms, NadamClip demonstrates better training stability while maintaining competitive prediction accuracy. It outperforms many optimizers, achieving an RMSE of 0.2644, MAE of 0.6595, and R² of 0.9743, while demonstrating excellent training stability. Compared to existing optimizer improvements, NadamClip leads the way by integrating gradient clipping with adaptive momentum estimation, moving beyond the traditional approach, where clipping primarily served as an external training control rather than an intrinsic part of the algorithm. Therefore, NadamClip offers a reliable solution for stabilizing and improving deep learning models in aquaculture and other fields