A survey of network anomaly detection techniques Ahmed, Mohiuddin; Naser Mahmood, Abdun; Hu, Jiankun
Journal of network and computer applications,
January 2016, 2016-01-00, Volume:
60
Journal Article
Peer reviewed
Information and Communication Technology (ICT) has a great impact on social wellbeing, economic growth and national security in todays world. Generally, ICT includes computers, mobile communication ...devices and networks. ICT is also embraced by a group of people with malicious intent, also known as network intruders, cyber criminals, etc. Confronting these detrimental cyber activities is one of the international priorities and important research area. Anomaly detection is an important data analysis task which is useful for identifying the network intrusions. This paper presents an in-depth analysis of four major categories of anomaly detection techniques which include classification, statistical, information theory and clustering. The paper also discusses research challenges with the datasets used for network intrusion detection.
•Maps different types of anomalies with network attacks.•Provides an up-to-date taxonomy of network anomaly detection.•Evaluates effectiveness of different categories of techniques.•Explores recent research related to publicly available network intrusion evaluation datasets.
Although the Internet of Things (IoT) can increase efficiency and productivity through intelligent and remote management, it also increases the risk of cyber-attacks. The potential threats to IoT ...applications and the need to reduce risk have recently become an interesting research topic. It is crucial that effective Intrusion Detection Systems (IDSs) tailored to IoT applications be developed. Such IDSs require an updated and representative IoT dataset for training and evaluation. However, there is a lack of benchmark IoT and IIoT datasets for assessing IDSs-enabled IoT systems. This paper addresses this issue and proposes a new data-driven IoT/IIoT dataset with the ground truth that incorporates a label feature indicating normal and attack classes, as well as a type feature indicating the sub-classes of attacks targeting IoT/IIoT applications for multi-classification problems. The proposed dataset, which is named TON_IoT, includes Telemetry data of IoT/IIoT services, as well as Operating Systems logs and Network traffic of IoT network, collected from a realistic representation of a medium-scale network at the Cyber Range and IoT Labs at the UNSW Canberra (Australia). This paper also describes the proposed dataset of the Telemetry data of IoT/IIoT services and their characteristics. TON_IoT has various advantages that are currently lacking in the state-of-the-art datasets: i) it has various normal and attack events for different IoT/IIoT services, and ii) it includes heterogeneous data sources. We evaluated the performance of several popular Machine Learning (ML) methods and a Deep Learning model in both binary and multi-class classification problems for intrusion detection purposes using the proposed Telemetry dataset.
Several recent studies have suggested Blockchain for Peer-to-Peer energy trading (P2P-ET) to achieve better security, privacy and fast payment settlement. Most of them however rely on either public ...Blockchains (which have low performance) or permissioned blockchains (which have low decentralization level and do not provide byzantine fault tolerance). Moreover, these solutions have limitations when capturing the business model of existing energy trading systems. This article proposes a Unified permissioned blockchain-based P2P-ET Architecture (UBETA) that integrates three different types of energy markets and provides a unified energy trading and payment settlement model. The UBETA system is based on an enterprise Ethereum Blockchain, known as Hyperledger Besu, and Istanbul Byzantine Fault Tolerance (IBFT) consensus algorithm. We compared the performance of the proposed IBFT-based system with three existing systems (i.e., Ethereum Clique, Ethereum Proof of Work and Hyperledger Fabric's Raft) using specific performance metrics (i.e., read/write transaction latency, read/write transaction throughput and fail rate). The experiments were carried out on a network size of up to 60 nodes and a real energy trading data set from the Western Australian energy market was used. The experiment results indicate that the IBFT-based system has 15x lower latency and nearly 2x throughput compared to existing Proof of Work based P2P-ET solutions. Moreover, the system provides better scalability and success rate than existing Raft based P2P-ET systems: the fail rate of the IBFT-based system only increased by 11% while that of Raft increased by 20% when increasing the number of nodes from 20 to 60. In addition, the proposed unified energy trading model provides lower latency and reduces the number of blockchain transactions compared to the non-unified counterpart.
Enabling global Internet access is challenging for cellular-based Internet of Things (IoT) due to the limited range of terrestrial network services. One viable solution is to deploy IoT over ...satellite systems for coverage extension. However, operating a hybrid satellite–terrestrial network might incur high satellite bandwidth consumption and excessive service latency. Aiming to reduce the content delivery latency from the Internet-connected gateway to the users, this article proposes a two-tier cache-enabled model with full-duplex transmissions where content caches are deployed at the satellite and ground station. A closed-form solution for the successful delivery probability (SDP) of the files is derived considering the requested content distributions and channel statistics. Then, the SDP performance under common caching policies can be evaluated. The results are also used to optimize cache placement under caching capacity constraints. Numerical results demonstrate the performance improvements of the proposed system over those of single-tier cache-aided and half-duplex transmission systems.
Hierarchical consensus protocols are one of the most promising approaches considered to improve the performance and scalability of blockchain-based peer-to-peer energy trading (P2P ET) systems. They ...allow a network to be divided into multiple clusters where each cluster can independently run its own local consensus on local transactions (i.e., transactions that belong to a single cluster) and also cooperate with other clusters for global transactions (i.e., transactions that affect more than one cluster). However, current solutions do not allow the parallel processing of local and global transactions, leading to low throughput and high latency. Moreover, they impose an additional burden on the leader node by assigning to it the responsibility for both local and global consensus. This paper proposes a hierarchical concurrent optimistic blockchain consensus protocol, called HiCoOB, which includes an optimistic local consensus and a deterministic global consensus that enables the concurrent execution of local and global transactions. Additionally, the local and the global consensus are coordinated by two distinct leaders, thereby reducing the overhead on the local leader node. HiCoOB also provides a synchronization method between local and global consensus to avoid the double spending of energy or money that arises from interdependent client transactions. Experimental results indicate that the proposed protocol achieves 87% higher throughput than the existing systems. Furthermore, it reduces the waiting time of the local transactions by a factor of 5x. Finally, HiCoOB has 10 times less overhead on the local leader node in terms of message exchange when compared to state-of-the-art systems.
Anomaly detection is an important data analysis task. It is used to identify interesting and emerging patterns, trends and anomalies from data. Anomaly detection is an important tool to detect ...abnormalities in many different domains including financial fraud detection, computer network intrusion, human behavioural analysis, gene expression analysis and many more. Recently, in the financial sector, there has been renewed interest in research on detection of fraudulent activities. There has been a lot of work in the area of clustering based unsupervised anomaly detection in the financial domain. This paper presents an in-depth survey of various clustering based anomaly detection technique and compares them from different perspectives. In addition, we discuss the lack of real world data and how synthetic data has been used to validate current detection techniques.
•Recently, in the financial sector, there has been renewed interest in research on detection of fraudulent activities.•This paper presents an in-depth survey of various clustering based anomaly detection techniques and compares them from different perspectives.•In addition, we discuss the lack of real world data and how synthetic data has been used to validate current detection techniques.
Cyber-Physical Systems (CPSs) enable Information Technology to be integrated with Operation Technology to efficiently monitor and manage the physical processes of various critical infrastructures. ...Recent incidents in cyber ecosystems have shown that CPSs are becoming increasingly vulnerable to complex attacks. These incidents often lead to sensing and actuation misbehaviour by illegal manipulations of data, which can severely impact the underlying physical processes of critical infrastructures. Current research acknowledges that IT-based security measures cannot entirely protect CPSs from such threats. Moreover, they are not designed to monitor the measurement level activities of physical processes, and they fail to mitigate blended cyberattacks, especially multi-stage and zero-day ones. This article addresses these limitations by proposing a framework, named UnSupervised Misbehaviour Detection (USMD), comprising a deep neural network that learns about a system's expected behaviour from data-driven representations. USMD can identify in real-time the attacks on CPSs by using the long-short term memory and Attention method for multi-sensor data. The USMD's performance is evaluated on various known data sets (i.e., ToN_IoT, SWaT, WADI and Gas pipeline datasets). The experimental results indicate that the superior performance of USMD compared with six state-of-the-art methods, which we implemented and extensively tested. USMD achieves F-scores of 0.9699 and 0.9702 on SWaT and WADI datasets, respectively.
With the large-scale deployment of smart meters worldwide, research in non-intrusive load monitoring (NILM) has seen a significant rise due to its dual use of real-time monitoring of end-user ...appliances and user-centric feedback of power consumption usage. NILM is a technique for estimating the state and the power consumption of an individual appliance in a consumer's premise using a single point of measurement device such as a smart meter. Although there are several existing NILM techniques, there is no meaningful and accurate metric to evaluate these NILM techniques for multi-state devices such as the fridge, heat pump, etc. In this paper, we demonstrate the inadequacy of the existing metrics and propose a new metric that combines both event classification and energy estimation of an operational state to give a more realistic and accurate evaluation of the performance of the existing NILM techniques. In particular, we use unsupervised clustering techniques to identify the operational states of the device from a labeled dataset to compute a penalty threshold for predictions that are too far away from the ground truth. Our work includes experimental evaluation of the state-of-the-art NILM techniques on widely used datasets of power consumption data measured in a real-world environment.
Malicious software (Malware) plays an important role in penetrating and extracting sensitive information. Based on dynamic program’s behavior monitoring, existing solutions have shown that the Hidden ...Markov Model (HMM) is efficient in detecting malware using sequences of API calls. However, an obfuscation technique could insert minimal data stealing code into a large set of legitimate instructions, which makes the detector ineffective. Additionally, existing solutions require a whole picture of a program’s actions, and hence a small chunk of activities is much harder to detect. Substantial performance degradation can occur during the detection when a long sequence of APIs is used. This paper proposes the Sub-Curve HMM feature extraction approach that focuses on matching subsets of activities from the running processes that potentially lead to data exfiltration incidents. Sequences of API calls are used to train HMMs and test the likelihood of matching to the model. Malicious and benign activities gain different matching scores over an adjoining set of API calls. By projecting a sequence of matching score into a curve, our approach discriminates malicious actions using discontinuities in the slope of the curve. The experimental results show that the proposed approach outperforms existing solutions in detecting six (6) families of malware: the detection accuracy of Sub-Curve HMM is over 94% compared to 83% for the baseline HMM approach and 73% for Information Gain.
The resource-constrained nature and large-scale adoption of Internet of Things (IoT) have a significant challenge for securing IoT applications. This necessitates a robust and lightweight security ...architecture and schemes as the existing traditional Internet security architecture and protocols require huge resources and lack of end-to-end security mechanism. In this research, a resource efficient end-to-end security scheme has been proposed by offloading computations and storage of security parameters to fog nodes in the vicinity. In addition, a symmetric-key payload encryption has been used to minimize the overhead of message communication in the resource-contested IoT environment. The analysis shows that the proposed scheme outperforms Transport Layer Security (TLS) in resource usage while it maintains equivalent authenticated end-to-end communication between communicating IoT nodes. The proposed end-to-end security scheme saves more communication bandwidth and incurs less overhead as compared to existing TLS-based security schemes. In particular, the proposed system uses less number of handshakes and achieves a decrease in the number of transmitted messages (approximately 184 bytes as compared to compared TSL message size of 332 bytes) for every handshake. Further, it has been demonstrated through experiments that the proposed security method incurs less overheads as compared to the TLS bandwidth consumption considering a single connection session during message subscription.
PDF
