Nonlinear Functional Modeling Using Neural Networks

References

• Asadi, E., Silva, M. G., Antunes, C. H., Dias, L., and Glicksman, L. (2014), “Multi-Objective Optimization for Building Retrofit: A Model Using Genetic Algorithm and Artificial Neural Network and an Application,” Energy and Buildings, 81, 444–456. DOI: 10.1016/j.enbuild.2014.06.009.
   Web of Science ®Google Scholar
• Ayinde, B. O., Inanc, T., and Zurada, J. M. (2019), “Regularizing Deep Neural Networks by Enhancing Diversity in Feature Extraction,” IEEE Transactions on Neural Networks and Learning Systems, 30, 2650–2661. DOI: 10.1109/TNNLS.2018.2885972.
   PubMed Web of Science ®Google Scholar
• Chan, W., Jaitly, N., Le, Q., and Vinyals, O. (2016), “Listen, Attend and Spell: A Neural Network for Large Vocabulary Conversational Speech Recognition,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4960–4964. DOI: 10.1109/ICASSP.2016.7472621.
   Google Scholar
• Chen, D., Hall, P., and Müller, H. (2012), “Single and Multiple Index Functional Regression Models with Nonparametric Link,” Annals of Statistics, 39, 1720–1747.
   Web of Science ®Google Scholar
• Chen, Z., and Huang, X. (2017), “End-to-End Learning for Lane Keeping of Self-Driving Cars,” in 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 1856–1860. DOI: 10.1109/IVS.2017.7995975.
   Google Scholar
• Conan-Guez, B., and Rossi, F. (2002), “Multi-Layer Perceptrons for Functional Data Analysis: A Projection Based Approach,” in ICANN.
   Google Scholar
• Dorffner, G. (1996), “Neural Networks for Time Series Processing,” Neural Network World, 6, 447–468.
   Google Scholar
• Eilers, P. H., Li, B., and Marx, B. D. (2009), “Multivariate Calibration with Single-Index Signal Regression,” Chemometrics and Intelligent Laboratory Systems, 96, 196–202. Chimiometrie 2007, Lyon, France, 29–30 November 2007.
   Web of Science ®Google Scholar
• Epifanio, I. (2008), “Shape Descriptors for Classification of Functional Data,” Technometrics, 50, 284–294. DOI: 10.1198/004017008000000154.
   Web of Science ®Google Scholar
• Fan, Y., James, G. M., and Radchenko, P. (2015), “Functional Additive Regression,” Annals of Statistics, 43, 2296–2325.
   Web of Science ®Google Scholar
• Ferraty, F. (2011), Recent Advances in Functional Data Analysis and Related Topics, Heidelberg: Springer.
   Google Scholar
• Ferraty, F., Goia, A., Salinelli, E., and Vieu, P. (2013), “Functional Projection Pursuit Regression,” TEST, 22, 293–320. DOI: 10.1007/s11749-012-0306-2.
   Web of Science ®Google Scholar
• Ferraty, F., and Romain, Y. (2011), The Oxford Handbook of Functional Data Analysis, Oxford: Oxford University Press.
   Google Scholar
• Ferraty, F., and Vieu, P. (2006), Nonparametric Functional Data Analysis: Theory and Practice (Springer Series in Statistics), Berlin, Heidelberg: Springer-Verlag.
   Google Scholar
• Geenens, G. (2011), “Curse of Dimensionality and Related Issues in Nonparametric Functional Regression,” Statistics Surveys, 5, 30–43. DOI: 10.1214/09-SS049.
   Google Scholar
• Graves, A., Mohamed, A., and Hinton, G. (2013), “Speech Recognition with Deep Recurrent Neural Networks,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645–6649.
   Google Scholar
• Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., and Wierstra, D. (2015), “Draw: A Recurrent Neural Network for Image Generation,” in Proceedings of the 32nd International Conference on Machine Learning, eds. F. Bach and D. Blei, PMLR, vol. 37, pp. 1462–1471. Available at https://proceedings.mlr.press/v37/gregor15.html
   Google Scholar
• Horváth, L., and Kokoszka, P. (2012), Inference for Functional Data with Applications, Springer Series in Statistics, New York: Springer.
   Google Scholar
• James, G., and Silverman, B. (2004), “Functional Adaptive Model Estimation,” Journal of the American Statistical Association, 100, 565–576. DOI: 10.1198/016214504000001556.
   Web of Science ®Google Scholar
• Joshi, A., Khosravy, M., and Gupta, N. (2020), Machine Learning for Predictive Analysis: Proceedings of ICTIS 2020, Lecture Notes in Networks and Systems. Singapore: Springer.
   Google Scholar
• Kanellopoulos, I., and Wilkinson, G. G. (1997), “Strategies and Best Practice for Neural Network Image Classification,” International Journal of Remote Sensing, 18, 711–725. DOI: 10.1080/014311697218719.
   Web of Science ®Google Scholar
• Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei, L. (2014), “Large-Scale Video Classification with Convolutional Neural Networks,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1725–1732.
   Google Scholar
• Khashei, M., and Bijari, M. (2010), “An Artificial Neural Network (p,d,q) Model for Timeseries Forecasting,” Expert Systems with Applications, 37, 479–489. DOI: 10.1016/j.eswa.2009.05.044.
   Web of Science ®Google Scholar
• Kokoszka, P., and Reimherr, M. (2018), Introduction to Functional Data Analysis, New York: Chapman and Hall/CRC.
   Google Scholar
• Li, B., and Song, J. (2017), “Nonlinear Sufficient Dimension Reduction for Functional Data,” Annals of Statistics, 45, 1059–1095.
   Web of Science ®Google Scholar
• Ludwig, O., Nunes, U., and Araujo, R. (2014), “Eigenvalue Decay: A New Method for Neural Network Regularization,” Neurocomputing, 124, 33–42. DOI: 10.1016/j.neucom.2013.08.005.
   Web of Science ®Google Scholar
• Ma, H., and Zhu, Z. (2016), “Continuously Dynamic Additive Models for Functional Data,” Journal of Multivariate Analysis, 150, 1–13. DOI: 10.1016/j.jmva.2016.05.003.
   Web of Science ®Google Scholar
• Maller, H., Wu, Y., and Yao, F. (2013), “Continuously Additive Models for Nonlinear Functional Regression,” Biometrika, 100, 607–622. DOI: 10.1093/biomet/ast004.
   Web of Science ®Google Scholar
• Maqueda, A. I., Loquercio, A., Gallego, G., Garc ía, N., and Scaramuzza, D. (2018), “Event-based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
   Google Scholar
• McLean, M. W., Hooker, G., Staicu, A., Scheipl, F., and Ruppert, D. (2014), “Functional Generalized Additive Models,” Journal of Computational and Graphical Statistics, 23, 249–269. DOI: 10.1080/10618600.2012.729985.
   PubMed Web of Science ®Google Scholar
• Morchid, M. (2018), “Parsimonious Memory Unit for Recurrent Neural Networks with Application to Natural Language Processing,” Neurocomputing, 314, 48–64. DOI: 10.1016/j.neucom.2018.05.081.
   Web of Science ®Google Scholar
• Morris, J. (2014), “Functional Regression,” Annual Review of Statistics and Its Application, 2, 321–359. DOI: 10.1146/annurev-statistics-010814-020413.
   Web of Science ®Google Scholar
• Muller, H., and Yao, F. (2008), “Functional Additive Models,” Journal of the American Statistical Association, 103, 1534–1544. DOI: 10.1198/016214508000000751.
   Web of Science ®Google Scholar
• Olver, P. J. (2019), Introduction to the Calculus of Variations, London: Imperial College Press.
   Google Scholar
• Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., and Cottrell, G. (2017), “A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction,” in Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia, IJCAI’17, AAAI Press, pp. 2627–2633. Available at https://dl.acm.org/10.5555/3172077.3172254
   Google Scholar
• Ramsay, J., and Silverman, B. (1997), Functional Data Analysis, Springer Series in Statistics, New York: Springer.
   Google Scholar
• Rao, A. R., Wang, Q., Wang, H., Khorasgani, H., and Gupta, C. (2020), “Spatio-Temporal Functional Neural Networks,” in 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), pp. 81–89. DOI: 10.1109/DSAA49011.2020.00020.
   Google Scholar
• Reimherr, M., Sriperumbudur, B., and Taoufik, B. (2017), “Optimal Prediction for Additive Function-on-Function Regression,” Electronic Journal of Statistics, 12, 4571–4601. DOI: 10.1214/18-EJS1505.
   Web of Science ®Google Scholar
• Reiss, P., Goldsmith, J., Shang, H. L., and Ogden, R. (2016), “Methods for Scalar-on-Function Regression,” International Statistical Review, 85, 228–249. DOI: 10.1111/insr.12163.
   PubMed Web of Science ®Google Scholar
• Rossi, F., and Conan-Guez, B. (2006), “Theoretical Properties of Projection based Multilayer Perceptrons with Functional Inputs,” Neural Processing Letters, 23, 55–70. DOI: 10.1007/s11063-005-3100-2.
   Web of Science ®Google Scholar
• Rossi, F., Conan-Guez, B., and Fleuret, F. (2002), “Functional Data Analysis with Multi Layer Perceptrons,” in Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN’02 (Cat. No.02CH37290) (Vol. 3), pp. 2843–2848.
   Google Scholar
• Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986), Learning Internal Representations by Error Propagation, Cambridge, MA: MIT Press, pp. 318–362.
   Google Scholar
• Seide, F., Li, G., and Yu, D. (2012), “Conversational Speech Transcription Using Context-Dependent Deep Neural Networks,” in Proceedings of the 29th International Coference on International Conference on Machine Learning, Edinburgh, Scotland, ICML’12, Madison, WI: Omnipress, pp. 437–440. Available at https://dl.acm.org/10.5555/3042573.3042574
   Google Scholar
• Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., and Wang, Z. (2016), “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-pixel Convolutional Neural Network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
   Google Scholar
• Singh, A., and Singh, K. K. (2017), “Satellite Image Classification using Genetic Algorithm Trained Radial Basis Function Neural Network, Application to the Detection of Flooded Areas,” Journal of Visual Communication and Image Representation, 42, 173–182. DOI: 10.1016/j.jvcir.2016.11.017.
   Web of Science ®Google Scholar
• Stinchcombe, M. (1999), “Neural Network Approximation of Continuous Functionals and Continuous Functions on Compactifications,” Neural Networks, 12, 467–477. DOI: 10.1016/s0893-6080(98)00108-7.
   PubMed Web of Science ®Google Scholar
• Stoker, T. M. (1986), “Consistent Estimation of Scaled Coefficients,” Econometrica, 54, 1461–1481. DOI: 10.2307/1914309.
   Web of Science ®Google Scholar
• Tuddenham, R., and Snyder, M. (1954), “Physical Growth of California Boys and Girls from Birth to Eighteen Years,” Publications in Child Development. University of California, Berkeley, 1, 183–364.
   PubMedGoogle Scholar
• Wang, Q., Wang, H., Gupta, C., Rao, A. R., and Khorasgani, H. (2020), “A Non-linear Function-on-Function Model for Regression with Time Series Data,” in 2020 IEEE International Conference on Big Data (Big Data), Los Alamitos, CA: IEEE Computer Society, pp. 232–239. Available at https://www.computer.org/csdl/proceedings-article/big-data/2020/09378087/1s64xQ8FPpK DOI: 10.1109/BigData50022.2020.9378087.
   Google Scholar
• Wang, Q., Zheng, S., Farahat, A., Serita, S., and Gupta, C. (2019), “Remaining Useful Life Estimation Using Functional Data Analysis,” in 2019 IEEE International Conference on Prognostics and Health Management (ICPHM), pp. 1–8. Available at https://www.semanticscholar.org/paper/Remaining-Useful-Life-Estimation-Using-Functional-Wang-Zheng/e3d5a020ae3e8c16c4e321878984898db800ee1a
   Google Scholar
• Wang, Q., Zheng, S., Farahat, A., Serita, S., Saeki, T., and Gupta, C. (2019), “Multilayer Perceptron for Sparse Functional Data,” in 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–10. DOI: 10.1109/IJCNN.2019.8851700.
   Google Scholar
• Wang, W., and Yang, Y. (2019), “Development of Convolutional Neural Network and Its Application in Image Classification: A Survey,” Optical Engineering, 58, 040901. DOI: 10.1117/1.OE.58.4.040901.
   Web of Science ®Google Scholar
• Wang, X., and Ruppert, D. (2013), “Optimal Prediction in an Additive Functional Model,” Statistica Sinica, 25, 567–589.
   Web of Science ®Google Scholar
• Yang, Y., Yuan, Y., Chatzimichailidis, A., Sloun, R., Lei, L., and Chatzinotas, S. (2020), “Proxsgd: Training Structured Neural Networks under Regularization and Constraints,” in International Conference on Learning Representations.
   Google Scholar
• Yao, F., and Muller, H. (2010), “Functional Quadratic Regression,” Biometrika, 97, 49–64. DOI: 10.1093/biomet/asp069.
   Web of Science ®Google Scholar
• Yue-Hei Ng, J., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., and Toderici, G. (2015), “Beyond Short Snippets: Deep Networks for Video Classification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4694–4702.
   Google Scholar
• Zhang, Z., Wang, H., Xu, F., and Jin, Y. (2017), “Complex-Valued Convolutional Neural Network and its Application in Polarimetric SAR Image Classification,” IEEE Transactions on Geoscience and Remote Sensing, 55, 7177–7188. DOI: 10.1109/TGRS.2017.2743222.
   Web of Science ®Google Scholar
• Zhu, H., Yao, F., and Zhang, H. (2013), “Structured Functional Additive Regression in Reproducing Kernel Hilbert Spaces,” Journal of the Royal Statistical Society, Series B, 76, 581–603. DOI: 10.1111/rssb.12036.
   Google Scholar