15.1 Introduction

As we have discussed in Chapter 7, the effective detection of insider threats requires monitoring mechanisms that are far more fine-grained than for external threat detection. These monitors must be efficiently and reliably deployable in the software environments where actions endemic to malicious insider missions are caught in a timely manner. Such environments typically include user-level applications, such as word processors, email clients, and web browsers for which reliable monitoring of internal events by conventional means is difficult.

To be able to detect the insider threats, we need to capture as accurately as possible not only the attributes of such insiders but also their behavior and communication. In Chapter 14, we argued that the data about the insiders arrive continuously and therefore could be modeled as data streams. Therefore, insider threat detection amounts to a stream data mining problem.

In this chapter, first, we will present related work with regard to insider threat and stream mining. Next, we will present related work with regard to big data and analytics perspective. The organization of this chapter is as follows. Related work on insider threat detection will be discussed in Section 15.2. Related work in stream mining will be discussed in Section 15.3. Big data issues will be discussed in Section 15.4. This chapter is summarized in Section 15.5.

15.2 Insider Threat Detection

Insider threat detection work has applied ideas from both intrusion detection and external threat detection ([SCHO01], [WANG03], [MAXI03], [SCHU02]). Supervised learning approaches collect system call trace logs, containing records of normal and anomalous behavior ([FORR96], [HOFM98], [NGUY03], [GAO04]), extract n-gram features from the collected data and use the extracted features to train classifiers. Text classification approaches treat each system call as a word in a bag-of-words model [LIAO02]. Various attributes of system calls, including arguments, object path, return value, and error status, have been exploited as features in various supervised learning methods ([KRUG03], [TAND03]).

Hybrid high-order Markov chain models detect anomalies by identifying a signature behavior for a particular user based on their command sequences [JU01]. The probabilistic anomaly detection (PAD) algorithm [STOL05] is a general-purpose algorithm for anomaly detection (in the windows environment) that assumes that anomalies or noise is a rare event in the training data. Masquerade detection is argued over by some individuals. A number of detection methods were applied to a dataset of “truncated” UNIX shell commands for 70 users [SCHO01]. Commands were collected using the UNIX acct auditing mechanism. For each user, a number of commands were gathered over a period of time. The detection methods were supervised by a multistep Markovian model and a combination of Bayes and Markov approaches. It was argued that the dataset was not appropriate for the masquerade detection task [MAXI03]. It was pointed out that the period of data gathering varied greatly from user to user (from several days to several months). Furthermore, commands were not logged in the order in which they were typed. Instead, they were coalesced when the application terminated the audit mechanism. This leads to the unfortunate consequence of possible faulty analysis of strict sequence data. Therefore, in this proposed work, we have not considered this dataset. These approaches differ from our supervised approach in that these learning approaches are static in nature and do not learn over evolving streams. In other words, stream characteristics of data are not explored further. Hence, static learning performance may degrade over time. On the other hand, our supervised approach will learn from evolving data streams. Our proposed work is based on supervised learning, and it can handle dynamic data or stream data well by learning from evolving streams. In anomaly detection, a one-class SVM (OCSVM) algorithm is used [STOL05]. OCSVM builds a model by training on normal data, and then it classifies test data as benign or anomalous based on geometric deviations from that normal training data. For masquerade detection, OCSVM training is as effective as two-class training [STOL05]. Investigations have been made into SVMs using binary features and frequency-based features. The OCSVM algorithm with binary features performed the best.

Recursive mining has been proposed to find frequent patterns [SZYM04]. OCSVM classifiers were used for masquerade detection after the patterns were encoded with unique symbols and all sequences rewritten with this new coding. To the best of our knowledge, there is no work that extends this OCSVM in a stream domain. Although our approach relies on OCSVM, it is extended to the stream domain so that it can cope with changes ([PARV11b], [PARV13]). Works have also explored unsupervised learning for insider threat detection, but only to static streams to our knowledge ([LIU05], [ESKI02]). Static graph-based anomaly detection (GBAD) approaches ([COOK07], [EBER07], [COOK00], [YAN02]) represent threat and nonthreat data as a graph and apply unsupervised learning to detect anomalies. The minimum description length (MDL) approach to GBAD has been applied to email, cell phone traffic, business processes, and cybercrime datasets ([STAN96], [KOWA08]). Our work builds upon GBAD and MDL to support dynamic, evolving streams ([PARV11a], [PARV13]).

Stream mining is a relatively new category of data mining research that applies to continuous data streams [FAN04]. In such settings, both supervised and unsupervised learning must be adaptive in order to cope with data whose characteristics change over time. There are two main approaches to adaptation: incremental learning ([DOMI01], [DAVI98]) and ensemble-based learning ([MASU10a], [MASU11a], [FAN04]). The past work has demonstrated that ensemble-based approaches are the more effective of the two, thus motivating our approach.

Ensembles have been used in the past to bolster the effectiveness of positive/negative classification ([MASU08], [MASU11a]). By maintaining an ensemble of K models that collectively vote on the final classification, the number of false negatives (FN) and false positives (FP) for a test set can be reduced. As better models are created, poorer models are discarded to maintain an ensemble of size exactly K. This helps the ensemble evolve with the changing characteristics of the stream and keeps the classification task tractable. A comparison of the above related works is summarized in Table 15.1. A more complete survey is available in [SALE08].

Insider threat detection work has utilized ideas from intrusion detection or external threat detection areas ([SCHO01], [WANG03]). For example, supervised learning has been applied to detect insider threats. System call traces from normal activity and anomaly data are gathered [HOFM98]; features are extracted from this data using n-gram and, finally, trained with classifiers. Authors [LIAO02] exploit the text classification idea in the insider threat domain where each system call is treated as a word in a bag-of-words model. System call, and related attributes, arguments, object path, return value, and error status of each system call are served as features in various supervised methods ([KRUG03], [TAND03]). A supervised model based on a hybrid high-order Markov chain model was adopted by researchers [JU01]. A signature behavior for a particular user based on the command sequences that the user executed is identified and then anomaly is detected.

Schonlau et al. [SCHO01] applied a number of detection methods to a dataset of “truncated” UNIX shell commands for 70 users. Commands were collected using the UNIX acct auditing mechanism. For each user, a number of commands were gathered over a period of time. The detection methods are supervised based on the multistep Markovian model and the combination of the Bayes and Markov approaches. Maxion et al. [MAXI03] argued that the Schonlau dataset was not appropriate for the masquerade detection task and created a new dataset using the Calgary dataset and applying the static supervised model.

These approaches differ from our work in the following ways. These learning approaches are static in nature and do not learn over evolving stream. In other words, stream characteristics of data are not explored further. Hence, static learner performance may degrade over time. On the other hand, our approach will learn from evolving data stream. We show that our approach is unsupervised and is as effective as a supervised model (incremental). Researchers have explored unsupervised learning [LIU05] for insider threat detection. However, this learning algorithm is static in nature. Although our approach is unsupervised, it learns at the same time from evolving stream over time, and more data will be used for unsupervised learning. In anomaly detection, an OCSVM algorithm is used. OCSVM builds a model from training on normal data and then classifies a test data as benign- or anomaly-based on geometric deviations from normal training data. Wang et al. [WANG03] showed for masquerade detection that OCSVM training is as effective as two-class training. The authors have investigated SVMs using binary features and frequency-based features. The one-class SVM algorithm with binary features performed the best. To find frequent patterns, Szymanski et al. [SZYM04] proposed recursive mining, encoded the patterns with unique symbols, and rewrote the sequence using this new coding. They used an OCSVM classifier for masquerade detection. These learning approaches are static in nature and do not learn over evolving stream.

15.3 Stream Mining

Stream mining is a new data mining area where data is continuous ([MASU11a], [MASU11b], [ALKH12a], [MASU11c], [MASU10b], [MASU13]). In addition, characteristics of data may change over time (concept drift). Here, supervised learning and unsupervised learning need to be adaptive to cope with changes. There are two ways through which adaptive learning can be developed: one is incremental learning and the other is ensemble-based learning. Incremental learning is used in user action prediction [DOMI01] but not for anomaly detection. Davidson et al. [DAVI98] introduced incremental probabilistic action modeling (IPAM), based on one-step command transition probabilities estimated from the training data. The probabilities were continuously updated with the arrival of a new command and modified with the usage of an exponential decay scheme. However, the algorithm is not designed for anomaly detection. Therefore, to the best of our knowledge, there is almost no work from other researchers that handles insider threat detection in the stream mining area. This is the first attempt to detect insider threat using stream mining ([PARV11a], [PARV11b], [PARV12b]).

Recently, unsupervised learning has been applied to detect insider threat in a data stream ([PARV13], [PARV11b]). This work does not consider sequence data for threat detection. Recall that sequence data is very common in an insider threat scenario. Instead, it considers data as a graph/vector and finds normative patterns and applies an ensemble-based technique to cope with changes. On the other hand, in our proposed approach, we consider user command sequences for anomaly detection and construct a quantized dictionary for normal patterns.

Users’ repetitive daily or weekly activities may constitute user profiles. For example, a user’s frequent command sequences may represent a normative pattern of that user. Finding normative patterns over dynamic data streams of unbounded length is challenging due to the requirement of a one-pass algorithm. For this, an unsupervised learning approach is used by exploiting a compressed/quantized dictionary to model common behavior sequences. This unsupervised approach needs to identify a normal user’s behavior in a single pass ([PARV12a], [PARV12b], [CHUA11]). One major challenge with these repetitive sequences is their variability in length. To combat this problem, we generate a dictionary that will contain any combination of possible normative patterns existing in the gathered data stream. In addition, we have incorporated the power of stream mining to cope with gradual changes. We have done experiments and shown that our USSL approach works well in the context of the concept drift and anomaly detection.

Our work ([PARV12a], [PARV12b]) differs from that of [CHUA11] in the following ways. First, the work in [CHUA11] focuses on dictionary construction to generate normal profiles. In other words, their work does not address the insider threat issue, which is our focus. Second, [CHUA11] does not consider ensemble-based techniques; our work exploits the ensemble-based technique with the combination of unsupervised learning (i.e., dictionary for benign sequences). Finally, when a number of users will grow, dictionary construction will become a bottleneck. The work of [CHUA11] does not consider the scalability issue; in our case, we address the scalability issue using a MapReduce framework.

In [PARV12a], an incremental approach is used. Ensemble-based techniques are not incorporated, but the literature used shows that ensemble-based techniques are more effective than those of the incremental variety for stream mining ([MASU10a], [MASU11a], [FAN04]). Therefore, our approach focuses on ensemble-based techniques [PARV12b].

Refer to Table 15.2 in which related approaches are unsupervised or supervised, and it has been explained whether they focus on concept-drift, detecting insider threat, and sequenced data from stream mining.

15.4 Big Data Techniques for Scalability

Stream data are continuously coming with high velocity and large size [ALKH12b]. This conforms to the characteristics of big data. “Big data” is data whose scale, diversity, and complexity require new architecture, techniques, algorithms, and analytics to manage it and extract value and hidden knowledge from it. Therefore, big data researchers are looking for tools to manage, analyze, summarize, visualize, and discover knowledge from the collected data in a timely manner and in a scalable fashion. Here, we will list some and discuss what problems we are solving in big data.

With regard to big data management, there are a number of techniques available that allow massively scalable data processing over grids of inexpensive commodity hardware such as the following. The Google File System ([CHAN06], [DEAN08]) is a scalable distributed file system that utilizes clusters of commodity hardware to facilitate data-intensive applications. The system is fault tolerant where the failure of the machine is normal due to the usage of commodity hardware. To cope with failure, data will replicate into multiple nodes. If one node is failing, the system will utilize the other node where replicated data exists.

MapReduce ([CHAN06], [DEAN08]) is a programming model that supports data-intensive applications in a parallel manner. The MapReduce paradigm supports map and reduce functions. Map generates a set of intermediate key and value pairs, and then the reduce function combines the results and deduces it. In fact, the map/reduce paradigm can solve many real-world problems as shown in ([CHAN06], [DEAN08]).

Hadoop ([BU10], [XU10], [ABOU09]) is an open-source apache project that supports the Google File System and the MapReduce paradigm. Hadoop is widely used to address the scalability issue along with MapReduce. For example, with the huge amount of semantic web datasets, Husain et al. ([HUSA09], [HUSA10], [HUSA11]) showed that Hadoop can be used to provide scalable queries. In addition, MapReduce technology has been exploited by the BioMANTA project [DING05] and SHARD (see also [BIOM] and [SHAR]).

Amazon developed Dynamo [DECA07], a distributed key-value store. Dynamo does not support master−slave architecture, which is supported by Hadoop. Nodes in Dynamo communicate via a gossip network. To achieve high availability and performance, Dynamo supports a model called eventual consistency by sacrificing rigorous consistency. In eventual consistency, updates will be propagated to nodes in the cluster asynchronously and a new version of the data will be produced for each update.

Google developed BigTable ([CHAN06], [CHAN08]), a column-oriented data storage system. BigTable utilizes the Google File System and Chubby [BURR06], a distributed lock service. BigTable is a distributed multidimensional sparse map based on row keys, column names, and time stamps.

Researchers [ABOU09] exploited the combined power of MapReduce and relational database technology. With regard to big data analytics, there are handfuls of works related to this topic. For example, on the one hand, some researchers focus on generic analytics tools to address the scalability issue. On the other hand, other researchers focus on specific analytics problems.

With regard to tools, Mahout is an open-source big data analytics tool to support classification, clustering, and a recommendation system for big data. In [CHU06], researchers customized well-known machine learning algorithms to take advantage of multicore machines and the MapReduce programming paradigm. MapReduce has been widely used for mining petabytes of data [MORE08].

With regard to specific problems, Al-Khateeb et al. [ALKH12b] and Haque et al. ([HAQU13a], [HAQU13b]) proposed scalable classification over evolving stream by exploiting the MapReduce and Hadoop frameworks. There are some research works on parallel boosting with MapReduce. Palit et al. [PALI12] proposed two parallel boosting algorithms, ADABOOST.PL and LOGITBOOST.PL.

15.5 Summary and Directions

Many of the learning techniques that have been proposed in the literature do not handle data streams. As a result, these techniques do not address the evolving nature of streams. Our goal is to adapt SVM techniques for data streams so that such techniques can be used to handle the insider threat problem. This chapter has discussed aspects of stream mining as well as applying stream mining for massive data.

In the ensuing chapters of Part III we will discuss the techniques we have designed for insider threat detection. In particular, we argue that data relevant to insider threats is typically accumulated over many years of organization and system operations, and is therefore best characterized as an unbounded data stream. We then show how learning techniques including ensemble-based learning can be used for insider threat detection.

References

[ABOU09]. A. Abouzeid, K. Bajda-Pawlikowski, D. J. Abadi, A. Rasin, A. Silberschatz, “HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads,” In Proceedings of the VLDB Endowment 2 (1), 922–933, 2009.

[ALKH12a]. T. Al-Khateeb, M. M. Masud, L. Khan, C. C. Aggarwal, J. Han, B. M. Thuraisingham, “Stream Classification with Recurring and Novel Class Detection Using Class-Based Ensemble,” In ICDM’2012: Proceedings of the 12th IEEE Conference on Data Mining, December 10−13, 2012, Brussels, Belgium, pp. 31–40, 2012.

[ALKH12b]. T. Al-Khateeb, M. M. Masud, L. Khan, B. M. Thuraisingham, “Cloud Guided Stream Classification Using Class-Based Ensemble.” In CLOUD’2012: Proceedings of the 5th IEEE Conference on Cloud Computing, June 24−29, Honolulu, HI, USA, pp. 694–701, 2012.

[BIOM]. http://www.itee.uq.edu.au/eresearch/projects/biomanta.

[BU10]. Y. Bu, B. Howe, M. Balazinska, M. Ernst, “Haloop: Efficient Iterative Data Processing on Large Clusters,” Proceedings of the VLDB Endowment 3 (1), 285–296, 2010.

[BURR06]. M. Burrows, “The Chubby Lock Service for Loosely-Coupled Distributed Systems,” In OSDI’06: Proceedings of the 7th Symposium on Operating Systems Design and Implementation, November 6−8, Seattle, Washington, D.C., pp. 335–350, 2006.

[CHAN06]. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, R. Gruber, “Bigtable: A Distributed Storage System for Structured Data (Awarded Best Paper),” In OSDI’06: 7th USENIX Symposium on Operating Systems Design and Implementation, November 6–8, Seattle, Washington, D.C., pp. 205–218, 2006.

[CHAN08]. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, R. E. Gruber, “BigTable: A Distributed Storage System for Structured Data,” ACM Transactions on Computer Systems 26 (2), Article #4, 2008.

[CHU06]. C. T. Chu, S. K. Kim, Y. A. Lin, Y. Yu, G. R. Bradski, A. Y. Ng, K. Olukotun, “Map-Reduce for Machine Learning on Multicore,” B. Sch¨ opf, J. C. Platt, T. Hoffman (eds.), Neural Information Processing Systems, MIT Press, Cambridge, MA, pp. 281–288, 2006.

[CHUA11]. S.-L. Chua, S. Marsland, H. W. Guesgen, “Unsupervised Learning of Patterns in Data Streams Using Compression and Edit Distance,” In IJCAI’2011: Proceedings of the 22nd International Joint Conference on Artificial Intelligence, July 16–22, Catalonia, Spain, pp. 1231–1236, 2011.

[COOK00]. D. J. Cook and L. B. Holder, “Graph-Based Data Mining,” IEEE Intelligent Systems 15 (2), 32–41, 2000.

[COOK07]. D. J. Cook and L. B. Holder, editors. Mining Graph Data, John Wiley & Sons, Inc., Hoboken, NJ, 2007.

[DAVI98]. B. D. Davison and H. Hirsh, “Predicting Sequences of User Actions. In Working Notes of the Joint Workshop on Predicting the Future: AI Approaches to Time Series Analysis.” 15th National Conference on Artificial Intelligence and Machine, AAAI Press, Madison, WI, pp. 5–12, 1998.

[DEAN08]. J. Dean and S. Ghemawat, “MapReduce: Simplified Data Processing on Large Clusters,” Communications of the ACM, 51(1), 107–113, 2008.

[DECA07]. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, W. Vogels, “Dynamo: Amazon’s Highly Available Key-Value Store,” T. C. Bressoud, M. F. Kaashoek (eds.), In SOSP’07: Proceedings of the 21st ACM Symposium on Operating Systems Principles, Oct. 14−17, Stevenson, Washington, D.C., pp. 205–220, 2007.

[DING05]. L. Ding, T. Finin, Y. Peng, P. P. da Silva, D. L. Mcguinness, “Tracking RDF Graph Provenance Using RDF Molecules,” Technical Report (TR-S-05-06), University of Maryland Baltimore County, 2005. http://ebiquity.umbc.edu/paper/html/id/240/.

[DOMI01]. P. Domingos and G. Hulten, “Catching Up with the Data: Research Issues in Mining Data Streams,” In DMKD’01: 2001 ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery, May 20, Santa Barbara, CA, USA, 2001.

[EBER07]. W. Eberle and L. B. Holder, “Mining for Structural Anomalies in Graph-Based Data,” In DMIN’07: Proceedings of International Conference on Data Mining, Las Vegas, NV, pp. 376–389, 2007.

[ESKI02]. E. Eskin, A. Arnold, M. Prerau, L. Portnoy, S. Stolfo, “A Geometric Framework for Unsupervised Anomaly Detection: Detecting Intrusions in Unlabeled data,” D. Barbar´, S. Jajodia (eds.), Applications of Data Mining in Computer Security, Chapter 4. Springer, New York, NY, 2002.

[FAN04]. W. Fan, “Systematic Data Selection to Mine Concept-Drifting Data Streams,” In Proceedings of ACM SIGKDD, Seattle, Washington, D.C., pp. 128–137, 2004.

[FORR96]. S. Forrest, S. A. Hofmeyr, A. Somayaji, T. A. Longstaf, “A Sense of Self for Unix Processes,” In Proceedings of the IEEE Symposium on Computer Security and Privacy (S&P), Oakland, CA, pp. 120–128, 1996.

[GAO04]. D. Gao, M. K. Reiter, D. Song, “On Gray-Box Program Tracking for Anomaly Detection,” In Proceedings of the USENIX Security Symposium, pp. 103–118, 2004.

[HAQU13a]. A. Haque, B. Parker, L. Khan, “Intelligent MapReduce Based Frameworks for Labeling Instances in Evolving Data Stream” In CloudCom’2013: Proceedings of the 5th International Conference on Cloud Computing Technology and Science, December 2−5, Bristol, UK, pp. 299–304, 2013.

[HAQU13b]. A. Haque, B. Parker, L. Khan, “Labeling Instances in Evolving Data Streams with Mapreduce,” BigData, Santa Clara, CA, pp. 387–394, 2013.

[HOFM98]. S. A. Hofmeyr, S. Forrest, A. Somayaji, “Intrusion Detection Using Sequences of System Calls,” Journal of Computer Security 6 (3), 151–180, 1998.

[HUSA09]. M. Husain, P. Doshi, L. Khan, B. Thuraisingham, “Storage and Retrieval of Large RDF Graph Using Hadoop and MapReduce,” In CloudCom’09: Proceedings of the 1st International Conference on Cloud Computing, pp. 680–686. Springer-Verlag, Berlin, 2009.

[HUSA10]. M. F. Husain, L. Khan, M. Kantarcioglu, B. Thuraisingham, “Data Intensive Query Processing for Large RDF Graphs Using Cloud Computing Tools,” In CLOUD’10: Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing, Washington, DC, pp. 1–10, 2010.

[HUSA11]. M. F. Husain, J. P. McGlothlin, M. M. Masud, L. R. Khan, B. M. Thuraisingham, “Heuristics-Based Query Processing for Large RDF Graphs Using Cloud Computing,” IEEE Transactions on Knowledge and Data Engineering 23 (9), 1312–1327, 2011.

[JU01]. W.-H. Ju and Y. Vardi, “A Hybrid High-Order Markov Chain Model for Computer Intrusion Detection,” Journal of Computational and Graphical Statistics 10 (2), 277–295, 2001.

[KOWA08]. E., Kowalski, T. Conway, S. Keverline, M. Williams, D. Cappelli, B. Willke, A. Moore, “Insider Threat Study: Illicit Cyber Activity in the Government Sector,” Technical Report, U.S. Department of Homeland Security, U.S. Secret Service, CERT, and the Software Engineering Institute (Carnegie Mellon University), 2008. http://resources.sei.cmu.edu/library/asset-view.cfm?assetID=52227.

[KRUG03]. C. Krugel, D. Mutz, F. Valeur, G. Vigna, “On the Detection of Anomalous System Call Arguments,” In ESORICS’03: Proceedings of the 8th European Symposium on Research in Computer Security, Gjovik, Norway, pp. 326–343, 2003.

[LIAO02]. Y. Liao and V. R. Vemuri, “Using Text Categorization Techniques for Intrusion Detection,” In Proceedings of the 11th USENIX Security Symposium, Berkeley, CA, pp. 51–59, 2002.

[LIU05]. A. Liu, C. Martin, T. Hetherington, S. Matzner, “A Comparison of System Call Feature Representations for Insider Threat Detection,” In IAW’05: Proceedings of the IEEE Information Assurance Workshop, West Point, NY, pp. 340–347, 2005.

[MASU08]. Masud, M. M., J. Gao, L. Khan, J. Han, B. Thuraisingham, “A Practical Approach to Classify Evolving Data Streams: Training with Limited Amount of Labeled Data,” In ICDM’08: Proceedings of the IEEE International Conference on Data Mining, West Point, NY, pp. 929–934, 2008.

[MASU10a]. M. M. Masud, Q. Chen, J. Gao, L. Khan, C. Aggarwal, J. Han, B. Thuraisingham, “Addressing Concept-Evolution in Concept-Drifting Data Streams,” In ICDM’10: Proceedings of the IEEE International Conference on Data Mining, Sydney, New South Wales, pp. 929–934, 2010.

[MASU10b]. M. M. Masud, J. Gao, L. Khan, J. Han, B. M. Thuraisingham, “Classification and Novel Class Detection in Data Streams with Active Mining,” In PKDD’10: Advances in Knowledge Discovery and Data Mining (Lecture Notes in Computer Science Series, vol. 6119, part 2), Springer, New York, NY, pp. 311–324, 2010.

[MASU11a]. M. M. Masud, J. Gao, L. Khan, J. Han, B. M. Thuraisingham, “Classification and Novel Class Detection in Concept-drifting Data Streams under Time Constraints,” IEEE Transactions on Knowledge and Data Engineering 23 (6), 859–874, 2011.

[MASU11b]. M. M., Masud, C. Woolam, J. Gao, L. Khan, J. Han, K. W. Hamlen, N. C. Oza, “Facing the Reality of Data Stream Classification: Coping with Scarcity of Labeled Data,” Knowledge and Information Systems 33 (1), 213–244, 2011.

[MASU11c]. M. M. Masud, T. Al-Khateeb, L. Khan, C. C. Aggarwal, J. Gao, J. Han, B. M. Thuraisingham, “Detecting Recurring and Novel Classes in Concept-Drifting Data Streams,” In ICDM’2011: Proceedings of the 11th IEEE Conference on Data Mining, December 11−14, Vancouver, BC, Canada, pp. 1176–1181, 2011.

[MASU13]. M. M., Masud, Q. Chen, L. Khan, C. C. Aggarwal, J. Gao, J. Han, A. N. Srivastava, N. C. Oza, “Classification and Adaptive Novel Class Detection of Feature-evolving Data Streams,” IEEE Transactions on Knowledge and Data Engineering 25 (7), 1484–1497, 2013.

[MAXI03]. R. A. Maxion, “Masquerade Detection Using Enriched Command Lines,” In DSN’03: Proceedings of the IEEE International Conference on Dependable Systems and Networks, San Francisco, CA, pp. 5–14, 2003.

[MORE08]. C. Moretti, K. Steinhaeuser, D. Thain, N. V. Chawla, “Scaling Up Classifiers to Cloud Computers,” In Proceedings of the 2008 8th IEEE International Conference on Data Mining, Washington, D.C., pp. 472–481, 2008.

[NGUY03]. N. Nguyen, P. Reiher, and G. H. Kuenning, “Detecting Insider Threats by Monitoring System Call Activity,” In IAW’03: Proceedings of the IEEE Information Assurance Workshop, West Point, NY, pp. 45–52, 2003.

[PALI12]. I. Palit and C. K. Reddy, “Scalable and Parallel Boosting with Mapreduce,” IEEE Transactions on Knowledge and Data Engineering 24 (10), 1904–1916, 2012.

[PARV11a]. P. Parveen, J. Evans, B. Thuraisingham, K. W. Hamlen, L. Khan, “Insider Threat Detection Using Stream Mining and Graph Mining,” In PASSAT’2011: Proceedings of the 3rd IEEE Conference on Privacy, Security, Risk and Trust, MIT, Boston, MA, USA, pp. 1102–1110, 2011.

[PARV11b]. P. Parveen, Z. R. Weger, B. Thuraisingham, K. W. Hamlen, L. Khan, “Supervised Learning for Insider Threat Detection Using Stream Mining,” In Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence, November 7–9, Boca Raton, FL, pp. 1032–1039, 2011.

[PARV12a]. P. Parveen and B. Thuraisingham, “Unsupervised Incremental Sequence Learning for Insider Threat Detection,” In ISI’2012: Proceedings of the. IEEE International Conference on Intelligence and Security, June, Washington, DC, pp. 141–143, 2012.

[PARV12b]. P. Parveen, N. McDaniel, B. Thuraisingham, L. Khan, “Unsupervised Ensemble Based Learning for Insider Threat Detection,” In PASSAT’2012: Proceedings of the 4th IEEE International Conference on Information Privacy, Security, Risk and Trus, September, Amsterdam, The Netherlands, pp. 718–727, 2012.

[PARV13]. P. Parveen, N. McDaniel, J. Evans, B. Thuraisingham, K. W. Hamlen, L. Khan, “Evolving Insider Threat Detection Stream Mining Perspective,” International Journal on Artificial Intelligence Tools 22 (5), 1360013, 2013.

[SALE08]. M. B., Salem, S. Herkshkop, S. J. Stolfo, “A Survey of Insider Attack Detection Research,” Insider Attack and Cyber Security 39, 69–90, 2008.

[SCHO01]. M. Schonlau, W. DuMouchel, W.-H. Ju, A. F. Karr, M. Theus, Y. Vardi, “Computer Intrusion: Detecting Masquerades,” Statistical Science 16 (1), 1–17, 2001.

[SCHU02]. E. E. Schultz, “A Framework for Understanding and Predicting Insider Attacks,” Computers and Security 21 (6), 526–531, 2002.

[SHAR]. http://www.cloudera.com/blog/2010/03/how-raytheon-esearchers-are-using-hadoop-to-build-a-scalable-distributed-triple-store.

[STAN96]. S. Staniford-Chen, S. Cheung, R. Crawford, M. Dilger, J. Frank, J. Hoagland, K. Levitt, C. Wee, R. Yip, D. Zerkle, “GrIDS—A Graph Based Intrusion Detection System for Large Networks,” In Proceedings of the 19th National Information Systems Security Conference, Baltimore, MD, pp. 361–370, 1996.

[STOL05]. S. J. Stolfo, F. Apap, E. Eskin, K. Heller, S. Hershkop, A. Honig, K. Svore, “A Comparative Evaluation of Two Algorithms for Windows Registry Anomaly Detection,” Journal of Computer Security 13 (4), 659–693, 2005.

[SZYM04]. B. K. Szymanski and Y. Zhang, “Recursive Data Mining for Masquerade Detection and Author Identification,” 13th Annual IEEE Information Assurance Workshop, Washington, DC, pp. 424–431, 2004.

[TAND03]. G. Tandon and P. Chan, “Learning Rules from System Call Arguments and Sequences for Anomaly Detection,” In DMSEC’03: Proceedings of the ICDM Workshop on Data Mining for Computer Security, Melbourne, FL, pp. 20–29, 2003.

[WANG 03]. H. Wang, W. Fan, P. S. Yu, J. Han, “Mining Concept-Drifting Data Streams Using Ensemble Classifiers,” In Proceedings of SIGKDD, Washington, DC, pp. 226–235, 2003.

[XU10]. Y. Xu, P. Kostamaa, L. Gao, “Integrating Hadoop and Parallel DBMS,” In SIGMOD’2010: Proceedings of the 2010 International Conference on Management of Data, New York, NY, pp. 969–974, 2010.

[YAN02]. X. Yan and J. Han, “gSpan: Graph-Based Substructure Pattern Mining,” In ICDM’02: Proceedings of the International Conference on Data Mining, Maebashi City, Japan, pp. 721–724, 2002.