Bibliography

1. E. H. L. Aarts and J. H. M. Korst. Simulated Annealing and Boltzmann Machines. Wiley, Chichester, England, 1989.

2. G. Alon, D. P. Kroese, T. Raviv, and R. Y. Rubinstein. Application of the cross-entropy method to the buffer allocation problem in a simulation-based environment. Annals of Operations Research, 134(1):137–151, 2005.

3. S. Asmussen and P. W. Glynn. Stochastic Simulation: Algorithms and Analyses. Springer, New York, 2007.

4. S. Asmussen and R. Y. Rubinstein. Steady state rare event simulation in queueing model and its complexity properties. In J. Dshalalow, editor, Advances in Queueing: Theory, Methods and Open Problems, volume I, pages 429–462. CRC Press, Boca Raton, FL, 1995.

5. R. Barlow and A. Marshall. Bounds for distribution with monotone hazard rate, I and II. Annals of Mathematical Statistics, 35:1234–1274, 1964.

6. R. Barlow and F. Proschan. Statistical Theory of Reliability and Life Testing. Holt, Rinehart & Wilson, New York, 1975.

7. A. G. Barto and R. S. Sutton. Reinforcement Learning. MIT Press, Cambridge, MA, 1998.

8. A. Ben-Tal and M. Taboule. Penalty functions and duality in stochastic programming via -divergence functionals. Mathematics of Operations Reseach, 12(2):224–240, 1987.

9. J. Blitzstein and P. Diaconis. A sequential importance sampling algorithm for generating random graphs with prescribed degrees. Internet Mathematics, 6:487–520, 2010.

10. V. S. Borkar and S. P. Meyn. The O.D.E. method for convergence of stochastic approximation and reinforcement learning. SIAM Journal on Control and Optimization, 38(2):447–469, 1999.

11. Z. I. Botev and D. P. Kroese. Global likelihood optimization via the cross-entropy method with an application to mixture models. In Proceedings of the 36th conference on Winter simulation, WSC '04, pages 529–535. Winter Simulation Conference, 2004.

12. Z. I. Botev and D. P. Kroese. An efficient algorithm for rare-event probability estimation, combinatorial optimization, and counting. Methodology and Computing in Applied Probability, 10(4):471–505, 2008.

13. Z. I. Botev and D. P. Kroese. Efficient Monte Carlo simulation via the generalized splitting method. Statistics and Computing, 22:1–16, 2012.

14. Z. I. Botev, P. L'Ecuyer, G. Rubino, R. Simard, and B. Tuffin. Static network reliability estimation via generalized splitting. INFORMS Journal on Computing, 6, 2012.

15. Z. I. Botev, D. P. Kroese, R. Y. Rubinstein, and P. L'Ecuyer. The cross-entropy method for optimization. In V. Govindaraju and C. R. Rao, editors, Machine Learning, volume 31 of Handbook of Statistics. Elsevier, Amsterdam, 2013.

16. Z. I. Botev, D. P. Kroese, and T. Taimre. Generalized cross-entropy methods. In Proceedings of RESIM06, pages 1–30, Bamberg, Germany, October 2006.

17. A. Boubezoula, S. Paris, and M. Ouladsinea. Application of the cross entropy method to the GLVQ algorithm. Pattern Recognition, 41(10):3173–3178, 2008.

18. Y. Burtin and B. Pittel. Asymptotic estimates of the reliability of a complex system. Engineering Cybernetics, 10(3):445–451, 1972.

19. L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst. Reinforcement Learning and Dynamic Programming Using Function Approximators. CRC Press, Boca Raton, FL, 2010.

20. H. Cancela, M. El Khadiri, and G. Rubino. Rare event analysis by Monte Carlo techniques in static models. In G. Rubino and B. Tuffin, editors, Rare Event Simulation Using Monte Carlo Methods, pages 145–170. Wiley, Chichester, England, 2009. Chapter 7.

21. H. Cancela and M. El Khadiri. On the RVR simulation algorithm for network reliability evaluation. IEEE Transactions on Reliability, 52(2):207–212, 2003.

22. H. Cancela, P. L'Ecuyer, M. Lee, G. Rubino, and B. Tuffin. Analysis and improvements of path-based methods for Monte Carlo reliability evaluation of static models. In J. Faulin, A. A. Juan, S. Martorell, and E. Ramirez-Marquez, editors, Simulation Methods for Reliability and Availability of Complex Systems, pages 65–84. Springer, London, 2010.

23. H. Cancela, P. L'Ecuyer, G. Rubino, and B. Tuffin. Combination of conditional Monte Carlo and approximate zero-variance importance sampling for network reliability estimation. In Proceedings of the 42nd Winter Simulation Conference, WSC '10, pages 1263–1274. Winter Simulation Conference, 2010.

24. F. Cérou and A. Guyader. Adaptive multilevel splitting for rare event analysis. Stochastic Analysis and Applications, 25(2):417–443, 2007.

25. J. C. C. Chan and D. P. Kroese. Improved cross-entropy method for estimation. Statistics and Computing, 22(5):1031–1040, 2012.

26. G. M. J-B. Chaslot, M. H. M. Winands, I. Szita, and H. J. van den Herik. Cross-entropy for Monte-Carlo tree search. ICGA Journal, 31(3):145–156, 2008.

27. J.-C. Chen, C.-K. Wen, C.-P. Li, and P. Ting. Cross-entropy optimization for the design of fiber Bragg gratings. Photonics Journal, IEEE, 4(5):1495–1503, oct. 2012.

28. Y. Chen, P. Diaconis, S. P. Holmes, and J. Liu. Sequential Monte Carlo methods for statistical analysis of tables. Journal of the American Statistical Association, 100:109–120, 2005.

29. K. Chepuri and T. Homem-de-Mello. Solving the vehicle routing problem with stochastic demands using the cross entropy method. Annals of Operations Research, 134(1):153–181, 2005.

30. N. Clisby. Efficient implementation of the pivot algorithm for self-avoiding walks. Journal of Statistical Physics, 140:349–392, 2010.

31. I. Cohen, B. Golany, and A. Shtub. Resource allocation in stochastic, finite-capacity, multi-project systems through the cross entropy methodology. Journal of Scheduling, 10(1):181–193, 2007.

32. T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. MIT Press, Cambridge, MA, third edition, 2009.

33. A. Costa, J. Owen, and D. P. Kroese. Convergence properties of the cross-entropy method for discrete optimization. Operations Research Letters, 35(5):573–580, 2007.

34. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, Hoboken, NJ, second edition, 2006.

35. M. Davis, G. Logemann, and D. Loveland. A machine program for theorem proving. Communications of the ACM, 5:394–397, 1962.

36. M. Davis and H. Putnam. A computing procedure for quantification theory. Journal of the ACM, 7:201–215, 1960.

37. P. T. de Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein. A tutorial on the cross-entropy method. Annals of Operations Research, 134(1):19–67, 2005.

38. T. Dean and P. Dupuis. Splitting for rare event simulation: A large deviations approach to design and analysis. Stochastic Processes and their Applications, 119(2):562–587, 2009.

39. T. Dean and P. Dupuis. The design and analysis of a generalized RESTART/DPR algorithm for rare event simulation. Annals of Operations Research, 189(1):63–102, 2011.

40. P. Dupuis, B. Kaynar, R. Reuvenstein, A. Ridder, and R. Vaisman. Counting with combined splitting and capture-recapture methods. Stochastic Models, 28:478–502, 2012.

41. T. Elperin, I. B. Gertsbakh, and M. Lomonosov. Estimation of network reliability using graph evolution models. IEEE Transactions on Reliability, 40(5):572–581, 1991.

42. T. Elperin, I. B. Gertsbakh, and M. Lomonosov. An evolution model for Monte Carlo estimation of equilibrium network renewal parameters. Probability in the Engineering and Informational Sciences, 6:457–469, 1992.

43. T. Elperin, I. B. Gertsbakh, and M. Lomonosov. On Monte Carlo estimates in network reliability. Probability in the Engineering and Informational Sciences, 8:245–265, 1994.

44. P. Erdös and A. Rényi. On the evolution of random graphs. In Publication of the Mathematical Institute of the Hungarian Academy of Sciences, pages 17–61, 1960.

45. D. Ernst, M. Glavic, G.-B. Stan, S. Mannor, and L. Wehenkel. The cross-entropy method for power system combinatorial optimization problems. In Proceedings of the 7th IEEE Power Engineering Society (IEEE-PowerTech 2007), pages 1290–1295, 2007.

46. G. E. Evans, J. M. Keith, and D. P. Kroese. Parallel cross-entropy optimization. In Proceedings of the 39th conference on Winter simulation, WSC '07, pages 2196–2202, Winter Simulation Conference, 2007.

47. G. Fishman. A Monte Carlo sampling plan for estimating network reliability. Operations Research, 34(4):581–594, 1980.

48. M. J. J. Garvels. The splitting method in rare-event simulation. PhD thesis, University of Twente, 2000.

49. M. J. J. Garvels and D. P. Kroese. A comparison of RESTART implementations. In Proceedings of the 30th conference on Winter simulation, WSC '98, pages 601–608, Winter Simulation Conference, 1998.

50. M. J. J. Garvels, D. P. Kroese, and J. C. W. van Ommeren. On the importance function RESTART simulation. European Transactions on Telecommunications, 13(4), 2002.

51. A. Gelman. and D. B. Rubin. Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7:457–511, 1992.

52. I. B. Gertsbakh and Y. Shpungin. Models of Network Reliability: Analysis, Combinatorics, and Monte Carlo. CRC Press, Boca Raton, FL, 2010.

53. A. Ghate and R. L. Smith. A dynamic programming approach to efficient sampling from Boltzmann distribution. Operations Research Letters, 36(6):665–668, 2008.

54. P. Glasserman, P. Heidelberger, P. Shahabuddin, and T. Zajic. A large deviations perspective on the efficiency of multilevel splitting. IEEE Transactions on Automatic Control, 43(12):1666–1679, 1998.

55. P. Glasserman, P. Heidelberger, P. Shahabuddin, and T. Zajic. Multilevel splitting for estimating rare event probabilities. Operations Research, 47(4):585–600, 1999.

56. V. Gogate and R. Dechter. Approximate counting by sampling the backtrack-free search space. In Proceedings of the 22nd national conference on Artificial intelligence—Volume 1, AAAI'07, pages 198–203. AAAI Press, Menlo Park, CA, 2007.

57. V. Gogate and R. Dechter. Samplesearch: Importance sampling in presence of determinism. Artificial Intelligence, 175(2):694–729, 2011.

58. E. Gryazina and B. Polyak. Randomized methods based on new Monte Carlo schemes for control and optimizations. Annals of Operations Research, 189(1):343–356, 2011.

59. K. P. Hui, N. Bean, M. Kraetzl, and D. P. Kroese. The cross-entropy method for network reliability estimation. Annals of Operations Research, 134(1):101–118, 2005.

60. M. Jerrum, L. G. Valiant, and V. V. Vazirani. Random generation of combinatorial structures from a uniform distribution. Theoretical Computer Science, 43:169–188, 1986.

61. L. P. Kaelbling, M. Littman, and A. W. Moore. Reinforcement learning—A survey. Journal of Artificial Intelligence Research, 4:237–285, May 1996.

62. H. Kahn and T. E. Harris. Estimation of particle transmission by random sampling. National Bureau of Standards Appl. Math. Series, 12:27–30, 1951.

63. J. N. Kapur and H. K. Kesavan. Entropy Optimization with Applications. Academic Press, Boston, 1992.

64. D. R. Karger. A randomized fully polynomial time approximation scheme for the all terminal network reliability problem. In Proceedings of the 27th annual ACM symposium on Theory of computing, STOC '95, pages 11–17, New York, NY, USA, 1995. ACM.

65. R. M. Karp, M. Luby, and N. Madras. Monte-Carlo approximation algorithms for enumeration problems. Journal of Algorithms, 10(3):429–448, 1989.

66. M. Kearns and S. Singh. Near-optimal reinforcement learning in polynomial time. Machine Learning, 49(2-3):209–232, 2002.

67. J. Keith and D. P. Kroese. Rare event simulation and combinatorial optimization using cross entropy: Sequence alignment by rare event simulation. In Proceedings of the 34th conference on Winter simulation, WSC '02, pages 320–327. Winter Simulation Conference, 2002.

68. D. E. Knuth. The Art of Computer Programming, Combinatorial Algorithms, Part 1, volume 4A. Addison-Wesley, Upper Saddle River, NJ, 2011.

69. R. P. Kothari and D. P. Kroese. Optimal generation expansion planning via the cross-entropy method. In Proceedings of the 41st conference on Winter simulation, WSC '09, pages 1482–1491. Winter Simulation Conference, 2009.

70. D. P. Kroese, K.-P. Hui, and S. Nariai. Network reliability optimization via the cross-entropy method. IEEE Transactions on Reliability, 56(2):275–287, 2007.

71. D. P. Kroese, S. Porotsky, and R. Y. Rubinstein. The cross-entropy method for continuous multi-extremal optimization. Methodology and Computing in Applied Probability, 8(3):383–407, 2006.

72. D. P. Kroese, R. Y. Rubinstein, and T. Taimre. Application of the cross-entropy method to clustering and vector quantization. Journal of Global Optimization, 37:137–157, 2007.

73. D. P. Kroese, T. Taimre, and Z. I. Botev. Handbook of Monte Carlo Methods. Wiley, New Jersey, 2011.

74. H. W. Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2:83–97, 1955.

75. H. W. Kuhn. Nonlinear programming: a historical view. ACM SIGMAP Bulletin, 31:6–18, June 1982.

76. P. L'Ecuyer, V. Demeres, and B. Tuffin. Splitting for rare-event simulation. In Proceedings of the 38th conference on Winter simulation, WSC '06, pages 137–148. Winter Simulation Conference, 2006.

77. P. L'Ecuyer, V. Demeres, and B. Tuffin. Rare-events, splitting, and quasi-Monte Carlo. ACM Transactions on Modeling and Computer Simulation, 12(2), 2007.

78. M. Liskiewicz, M. Ogiharaba, and S. Toda. The complexity of counting self-avoiding walks in subgraphs of two-dimensional grids and hypercubes. Theoretical Computer Science, 304:129–156, 2003.

79. J. S. Liu. Monte Carlo Strategies in Scientific Computing. Springer, New York, 2001.

80. Z. Liu, A. Doucet, and S. S. Singh. The cross-entropy method for blind multiuser detection. In Proceedings 2004 IEEE International Symposium on Information Theory, page 510, Chicago, IL, 2004.

81. A. Lörincza, Z. Palotaia, and G. Szirtesb. Spike-based cross-entropy method for reconstruction. Neurocomputing, 71(16-18):3635–3639, 2008.

82. N. Madras and A. D. Sokal. The pivot algorithm: A highly efficient Monte Carlo method for the self-avoiding walk. Journal of Statistical Physics, 50(1):109–186, 1988.

83. S. Mannor, R. Y. Rubinstein, and Y. Gat. The cross-entropy method for fast policy search. In Proceedings of the 20th International Conference on Machine Learning, pages 512–519, AAAI Press, Menlo Park, CA, 2003.

84. L. Margolin. On the convergence of the cross-entropy method. Annals of Operations Research, 134(1):201–214, 2005.

85. V. B. Melas. On the efficiency of the splitting and roulette approach for sensitivity analysis. In Proceedings of the 29th conference on Winter simulation, WSC '97, pages 269–274, Winter Simulation Conference, 1997.

86. I. Menache and S. Mannor. Basis function adaptation in temporal difference reinforcement learning. Annals of Operations Research, 134(1):215–238, 2005.

87. P. Metzner, C. Schutte, and E. Vanden-Eijnden. Illustration of transition path theory on a collection of simple examples. The Journal of Chemical Physics, 125(8):084–110, 2006.

88. M. Mitzenmacher and E. Upfal. Probability and Computing: Randomized Algorithms and Probabilistic Analysis. Cambridge University Press, Cambridge, 2005.

89. P. Del Moral. Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Probability and Its Applications. Springer, New York, 2004.

90. R. Motwani and R. Raghavan. Randomized Algorithms. Cambridge University Press, Cambridge, 1995.

91. M. S. Nikulin. Chi-squared test for normality. Proceedings of the International Vilnius Conference on Probability Theory and Mathematical Statistics, Volume 2:119–122, 1973.

92. C. H. Papadimitriou and K. Steiglitz. Combinatorial Optimization: Algorithms and Complexity. Prentice Hall, Englewood Cliffs, NJ, 1982.

93. V. Pihur, S. Datta, and S. Datta. Weighted rank aggregation of cluster validation measures: A Monte Carlo cross-entropy approach. Bioinformatics, 23(13):1607–1615, 2007.

94. M. A. Pincus. A closed form selection of certain programming problems. Operations Research, 16:690–694, 1968.

95. M. Puterman. Markov Decision Processes. Wiley, New York, 1994.

96. B. Roberts and D. P. Kroese. Estimating the number of s-t paths in a graph. Journal of Graph Algorithms and Applications, 11(1):195–214, 2007.

97. M. N. Rosenbluth and A. W. Rosenbluth. Monte Carlo calculation of the average extension of molecular chains. Journal of Chemical Physics, 23(2):356–359, 1955.

98. M. T. Rosenstein and A. G. Barto. Robot weightlifting by direct policy search. In Proceedings of the 17th International Joint Conference on Artificial Intelligence—Volume 2, IJCAI'01, pages 839–846, Morgan Kaufmann Publishers, San Francisco (CA), 2001.

99. S. M. Ross. Simulation. Elsevier Academic Press, Amsterdam, fourth edition, 2006.

100. R. Y. Rubinstein. Optimization of computer simulation models with rare events. European Journal of Operational Research, 99(1):89–112, 1997.

101. R. Y. Rubinstein. The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability, 1(2):127–190, 1999.

102. R. Y. Rubinstein. Combinatorial optimization, cross-entropy, ants and rare events. In S. Uryasev and P. M. Pardalos, editors, Stochastic Optimization: Algorithms and Applications, pages 304–358. Kluwer, Dordrecht, 2001.

103. R. Y. Rubinstein. A stochastic minimum cross-entropy method for combinatorial optimization and rare-event estimation. Methodology and Computing in Applied Probability, 7(1):1–46, 2005.

104. R. Y. Rubinstein. The Gibbs cloner for combinatorial optimization, counting and sampling. Methodology and Computing in Applied Probability, 11(2):491–549, 2009.

105. R. Y. Rubinstein. Randomized algorithms with splitting: Why the classic randomized algorithms do not work and how to make them work. Methodology and Computing in Applied Probability, 12(1):1–41, 2010.

106. R. Y. Rubinstein and P. W. Glynn. How to deal with the curse of dimensionality of likelihood ratios in Monte Carlo simulations. Stochastic Models, 25(4):547–568, 2009.

107. R. Y. Rubinstein and D. P. Kroese. The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer, New York, 2004.

108. R. Y. Rubinstein and D. P. Kroese. Simulation and the Monte Carlo Method. Wiley, Hoboken, NJ, second edition, 2008.

109. R. Y. Rubinstein, Z. Botev, and R. Vaisman. Hanging edges for fast reliability estimation. Private Communication, 2012.

110. F. J. Samaniego. On closure of the IFR under formation of coherent systems. IEEE Transactions on Reliability, 34:69–72, 1985.

111. A. Sani. Stochastic Modelling and Intervention of the Spread of HIV/AIDS. PhD thesis, The University of Queensland, Brisbane, 2009.

112. A. Sani and D. P. Kroese. Controlling the number of HIV infectives in a mobile population. Mathematical Biosciences, 213(2):103–112, 2008.

113. G. A. F. Seber. The effect of trap response on tag recapture estimates. Biometrics, 26:13–22, 1970.

114. G. A. F. Seber. Estimation of Animal Abundance and Related Parameters. Blackburn Press, Caldwell, NJ, second edition, 2002.

115. D. Siegmund. Importance sampling in the Monte Carlo study of sequential tests. Annals of Statististics, 4:673–684, 1976.

116. Z. Szabó, B. Póczos, and A. Lörinc. Cross-entropy optimization for independent process analysis. In Independent Component Analysis and Blind Signal Separation, Lecture Notes in Computer Science, Volume 3889, pages 909–916, Springer, Heidelberg, 2006.

117. J. N. Tsitsiklis. Asynchronous stochastic approximation and Q-learning. Machine Learning, 16:185–202, 1994.

118. A. Ünveren and A. Acan. Multi-objective optimization with cross entropy method: Stochastic learning with clustered pareto fronts. In Proceedings of the IEEE Congress on Evolutionary Computation, pages 3065–3071, 2007.

119. L. G. Valiant. The complexity of computing the permanent. Theoretical Computer Science, 8:189–201, 1979.

120. L. G. Valiant. The complexity of enumeration and reliability problems. SIAM Journal on Computing, 8:410–421, 1979.

121. E. J. J. van Rensburg. Monte Carlo methods for the self-avoiding walk. Journal of Physics A: Mathematical and Theoretical, 42(32):323001, 2009.

122. M. Villén-Altimirano and J. Villén-Altimirano. RESTART: A method for accelerating rare event simulations. In Proceedings of the 13th International Teletraffic Congress, Performance and Control in ATM, pages 71–76, June 1991.

123. M. Villén-Altimirano and J. Villén-Altimirano. RESTART: A straightforward method for fast simulation of rare events. In Proceedings of the 26th conference on Winter simulation, WSC '94, pages 282–289, Winter Simulation Conference, 1994.

124. M. Villén-Altimirano and J. Villén-Altimirano. About the efficiency of RESTART. In Proceedings of the 1999 RESIM Workshop, pages 99–128, University of Twente, the Netherlands, 1999.

125. C. Watkins. Learning from Delayed Rewards. PhD thesis, Cambridge University, 1989.

126. W. Wei and B. Selman. A new approach to model counting. In Proceedings of the 8th international conference on Theory and Applications of Satisfiability Testing, SAT'05, Lecture Notes in Computer Science, Volume 3569, pages 324–339, Springer, Heidelberg, 2005.

127. R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3-4,):229–256, 1992.

128. Y. Wu and C. Fyfe. Topology perserving mappings using cross entropy adaptation. In Proceedings of the 7th WSEAS International Conference on Artificial intelligence, knowledge engineering and data bases, AIKED'08, pages 176–181, World Scientific and Engineering Academy and Society, Stevens Point, WI, 2008.