Cover image for Machine Learning and Big Data with kdb+/q

Machine Learning and Big Data with kdb+/q

Author Jan Novotny , Paul A. Bilokon , Aris Galiotos
Release Date: 2019/12/01
Topic:
0%
45
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Book Description

Upgrade your programming language to more effectively handle high-frequency data

Machine Learning and Big Data with KDB+/Q offers quants, programmers and algorithmic traders a practical entry into the powerful but non-intuitive kdb+ database and q programming language. Ideally designed to handle the speed and volume of high-frequency financial data at sell- and buy-side institutions, these tools have become the de facto standard; this book provides the foundational knowledge practitioners need to work effectively with this rapidly-evolving approach to analytical trading.

The discussion follows the natural progression of working strategy development to allow hands-on learning in a familiar sphere, illustrating the contrast of efficiency and capability between the q language and other programming approaches. Rather than an all-encompassing “bible”-type reference, this book is designed with a focus on real-world practicality ­to help you quickly get up to speed and become productive with the language.

  • Understand why kdb+/q is the ideal solution for high-frequency data
  • Delve into “meat” of q programming to solve practical economic problems
  • Perform everyday operations including basic regressions, cointegration, volatility estimation, modelling and more
  • Learn advanced techniques from market impact and microstructure analyses to machine learning techniques including neural networks

The kdb+ database and its underlying programming language q offer unprecedented speed and capability. As trading algorithms and financial models grow ever more complex against the markets they seek to predict, they encompass an ever-larger swath of data ­– more variables, more metrics, more responsiveness and altogether more “moving parts.”

Traditional programming languages are increasingly failing to accommodate the growing speed and volume of data, and lack the necessary flexibility that cutting-edge financial modelling demands. Machine Learning and Big Data with KDB+/Q opens up the technology and flattens the learning curve to help you quickly adopt a more effective set of tools.   

Table of Contents

  1. Cover
  2. Preface
    1. HISTORY OF kdb+ AND q
    2. MOTIVATION FOR THIS BOOK
    3. CODE STRUCTURE
    4. STRUCTURE OF THE BOOK
    5. PREREQUISITES
    6. NOTES
  3. About the Authors
  4. PART One: Language Fundamentals
    1. CHAPTER 1: Fundamentals of the q Programming Language
      1. 1.1 THE (NOT SO VERY) FIRST STEPS IN q
      2. 1.2 ATOMS AND LISTS
      3. 1.3 BASIC LANGUAGE CONSTRUCTS
      4. 1.4 BASIC OPERATORS
      5. 1.5 DIFFERENCE BETWEEN STRINGS AND SYMBOLS
      6. 1.6 MATRICES AND BASIC LINEAR ALGEBRA IN q
      7. 1.7 LAUNCHING THE SESSION: ADDITIONAL OPTIONS
      8. 1.8 SUMMARY AND HOW-TO'S
    2. CHAPTER 2: Dictionaries and Tables: The q Fundamentals
      1. 2.1 DICTIONARY
      2. 2.2 TABLE
      3. 2.3 THE TRUTH ABOUT TABLES
      4. 2.4 KEYED TABLES ARE DICTIONARIES
      5. 2.5 FROM A VECTOR LANGUAGE TO AN ALGEBRAIC LANGUAGE
    3. CHAPTER 3: Functions
      1. 3.1 NAMESPACE
      2. 3.2 THE SIX ADVERBS
      3. 3.3 APPLY
      4. 3.4 PROTECTED EVALUATIONS
      5. 3.5 VECTOR OPERATIONS
      6. 3.6 CONVENTION FOR USER-DEFINED FUNCTIONS
    4. CHAPTER 4: Editors and Other Tools
      1. 4.1 CONSOLE
      2. 4.2 JUPYTER NOTEBOOK
      3. 4.3 GUIs
      4. 4.4 IDEs: INTELLIJ IDEA
      5. 4.5 CONCLUSION
    5. CHAPTER 5: Debugging q Code
      1. 5.1 INTRODUCTION TO MAKING IT WRONG: ERRORS
      2. 5.2 DEBUGGING THE CODE
      3. 5.3 DEBUGGING SERVER-SIDE
      4. NOTE
  5. PART Two: Data Operations
    1. CHAPTER 6: Splayed and Partitioned Tables
      1. 6.1 INTRODUCTION
      2. 6.2 SAVING A TABLE AS A SINGLE BINARY FILE
      3. 6.3 SPLAYED TABLES
      4. 6.4 PARTITIONED TABLES
      5. 6.5 CONCLUSION
      6. NOTE
    2. CHAPTER 7: Joins
      1. 7.1 COMMA OPERATOR
      2. 7.2 JOIN FUNCTIONS
      3. 7.3 ADVANCED EXAMPLE: RUNNING TWAP
    3. CHAPTER 8: Parallelisation
      1. 8.1 PARALLEL VECTOR OPERATIONS
      2. 8.2 PARALLELISATION OVER PROCESSES
      3. 8.3 MAP-REDUCE
      4. 8.4 ADVANCED TOPIC: PARALLEL FILE/DIRECTORY ACCESS
    4. CHAPTER 9: Data Cleaning and Filtering
      1. 9.1 PREDICATE FILTERING
      2. 9.2 DATA CLEANING, NORMALISING AND APIs
    5. CHAPTER 10: Parse Trees
      1. 10.1 DEFINITION
      2. 10.2 FUNCTIONAL QUERIES
    6. CHAPTER 11: A Few Use Cases
      1. 11.1 ROLLING VWAP
      2. 11.2 WEIGHTED MID FOR N LEVELS OF AN ORDER BOOK
      3. 11.3 CONSECUTIVE RUNS OF A RULE
      4. 11.4 REAL-TIME SIGNALS AND ALERTS
  6. PART Three: Data Science
    1. CHAPTER 12: Basic Overview of Statistics
      1. 12.1 HISTOGRAM
      2. 12.2 FIRST MOMENTS
      3. 12.3 HYPOTHESIS TESTING
    2. CHAPTER 13: Linear Regression
      1. 13.1 LINEAR REGRESSION
      2. 13.2 ORDINARY LEAST SQUARES
      3. 13.3 THE GEOMETRIC REPRESENTATION OF LINEAR REGRESSION
      4. 13.4 IMPLEMENTATION OF THE OLS
      5. 13.5 SIGNIFICANCE OF PARAMETERS
      6. 13.6 HOW GOOD IS THE FIT: R2
      7. 13.7 RELATIONSHIP WITH MAXIMUM LIKELIHOOD ESTIMATION AND AIC WITH SMALL SAMPLE CORRECTION
      8. 13.8 ESTIMATION SUITE
      9. 13.9 COMPARING TWO NESTED MODELS: TOWARDS A STOPPING RULE
      10. 13.10 IN‐/OUT‐OF‐SAMPLE OPERATIONS
      11. 13.11 CROSS‐VALIDATION
      12. 13.12 CONCLUSION
    3. CHAPTER 14: Time Series Econometrics
      1. 14.1 AUTOREGRESSIVE AND MOVING AVERAGE PROCESSES
      2. 14.2 STATIONARITY AND GRANGER CAUSALITY
      3. 14.3 VECTOR AUTOREGRESSION
    4. CHAPTER 15: Fourier Transform
      1. 15.1 COMPLEX NUMBERS
      2. 15.2 DISCRETE FOURIER TRANSFORM
      3. 15.3 ADDENDUM: QUATERNIONS
      4. 15.4 ADDENDUM: FRACTALS
    5. CHAPTER 16: Eigensystem and PCA
      1. 16.1 THEORY
      2. 16.2 ALGORITHMS
      3. 16.3 IMPLEMENTATION OF EIGENSYSTEM CALCULATION
      4. 16.4 THE DATA MATRIX AND THE PRINCIPAL COMPONENT ANALYSIS
      5. 16.5 IMPLEMENTATION OF PCA
      6. 16.6 APPENDIX: DETERMINANT
    6. CHAPTER 17: Outlier Detection
      1. 17.1 LOCAL OUTLIER FACTOR
    7. CHAPTER 18: Simulating Asset Prices
      1. 18.1 STOCHASTIC VOLATILITY PROCESS WITH PRICE JUMPS
      2. 18.2 TOWARDS THE NUMERICAL EXAMPLE
      3. 18.3 CONCLUSION
  7. PART Four: Machine Learning
    1. CHAPTER 19: Basic Principles of Machine Learning
      1. 19.1 NON-NUMERIC FEATURES AND NORMALISATION
      2. 19.2 ITERATION: CONSTRUCTING MACHINE LEARNING ALGORITHMS
    2. CHAPTER 20: Linear Regression with Regularisation
      1. 20.1 BIAS–VARIANCE TRADE-OFF
      2. 20.2 REGULARISATION
      3. 20.3 RIDGE REGRESSION
      4. 20.4 IMPLEMENTATION OF THE RIDGE REGRESSION
      5. 20.5 LASSO REGRESSION
      6. 20.6 IMPLEMENTATION OF THE LASSO REGRESSION
    3. CHAPTER 21: Nearest Neighbours
      1. 21.1 k -Nearest Neighbours Classifier
      2. 21.2 PROTOTYPE CLUSTERING
      3. 21.3 FEATURE SELECTION: LOCAL NEAREST NEIGHBOURS APPROACH
    4. CHAPTER 22: Neural Networks
      1. 22.1 THEORETICAL INTRODUCTION
      2. 22.2 IMPLEMENTATION OF NEURAL NETWORKS
      3. 22.3 EXAMPLES
      4. 22.4 POSSIBLE SUGGESTIONS
    5. CHAPTER 23: AdaBoost with Stumps
      1. 23.1 BOOSTING
      2. 23.2 DECISION STUMPS
      3. 23.3 ADABOOST
      4. 23.4 IMPLEMENTATION OF ADABOOST
      5. 23.5 RECOMMENDATION FOR READERS
    6. CHAPTER 24: Trees
      1. 24.1 INTRODUCTION TO TREES
      2. 24.2 REGRESSION TREES
      3. 24.3 CLASSIFICATION TREE
      4. 24.4 MISCELLANEOUS
      5. 24.5 IMPLEMENTATION OF TREES
    7. CHAPTER 25: Forests
      1. 25.1 BOOTSTRAP
      2. 25.2 BAGGING
      3. 25.3 IMPLEMENTATION
    8. CHAPTER 26: Unsupervised Machine Learning: The Apriori Algorithm
      1. 26.1 APRIORI ALGORITHM
      2. 26.2 IMPLEMENTATION OF THE APRIORI ALGORITHM
    9. CHAPTER 27: Processing Information
      1. 27.1 INFORMATION RETRIEVAL
      2. 27.2 INFORMATION AS FEATURES
    10. CHAPTER 28: Towards AI – Monte Carlo Tree Search
      1. 28.1 MULTI-ARMED BANDIT PROBLEM
      2. 28.2 MONTE CARLO TREE SEARCH
      3. 28.3 MONTE CARLO TREE SEARCH IMPLEMENTATION – TIC-TAC-TOE
      4. 28.4 MONTE CARLO TREE SEARCH – ADDITIONAL COMMENTS
    11. CHAPTER 29: Econophysics: The Agent‐Based Computational Models
      1. 29.1 AGENT‐BASED MODELLING
      2. 29.2 ISING AGENT‐BASED MODEL FOR FINANCIAL MARKETS
      3. 29.3 CONCLUSION
    12. CHAPTER 30: Epilogue: Art
  8. Bibliography
  9. Index
  10. End User License Agreement
3.145.77.114