Crowd management is crucial for countries and organizations as it can lead to severe consequences or serious safety concerns. Most of the existing research focus on addressing limited crowd management issues, namely crowd counting, density estimation, localization, and behavior monitoring. Furthermore, the generated incidents' alerts are mostly not interpretable and remediable. Therefore, there is no comprehensive solution that addresses all these issues. This research proposes a comprehensive intelligence-based crowd management framework that employs anomaly rules to monitor, predict, and detect crowd accidents and help in providing quick response. The suggested crowd intelligence framework addresses all crowd management issues. The use case chosen for this framework is the management of crowds of pilgrims in Umrah Holy event. The proposed framework is then implemented and evaluated with respect to efficiency, scalability, interpretability, remediability, and the number of false positive, true positive, and false negative alerts. In addition, the suggested framework is compared with other recent related work in terms of supporting crowd management issues. The design of the proposed framework and implementation are then fine-tuned in light of the evaluation results. The results and findings of this research can be extended to manage crowds at any event.