EDI-Graphic: A Tool To Study Parameter Discrimination and Confirm Identifiability in Black-Box Models, and to Select Data-Generating Machines

References

• Birgé, L. (2006), “Model Selection via Testing: An Alternative to Penalized Maximum Likelihood Estimators,” Annales de l’Institut Henri Poincaré, 42, 273–325.
   Google Scholar
• Breiman, L. (2001), “Statistical Modeling: The Two Cultures,” Statistical Science, 16, 199–231. DOI: 10.1214/ss/1009213726.
   Web of Science ®Google Scholar
• Breiman, L. (2002), “Looking Inside the Black Box,” available at https://www.stat.berkeley.edu/users/breiman/wald2002-2.pdf
   Google Scholar
• Csörgo, M., and Horvath, L. (1997), Limit Theorems in Change-Point Analysis, New York: Wiley.
   Google Scholar
• Dempster, A. P., and Schatzoff, M. (1965), “Expected Significance Level as a Sensibility Index for Test Statistics,” Journal of the American Statistical Association, 60, 420–436. DOI: 10.1080/01621459.1965.10480802.
   Web of Science ®Google Scholar
• Fukumizu, K. (2003), “Likelihood Ratio of Non-identifiable Models and Multilayer Neural Networks,” Annals of Statistics, 31, 833–851.
   Web of Science ®Google Scholar
• Fukumizu, K., and Amari, S. (2000), “Local Minima and Plateaus in Hierarchical Structures of Multilayer Perceptions,” Neural Networks, 13, 317–327. DOI: 10.1016/s0893-6080(00)00009-5.
   PubMed Web of Science ®Google Scholar
• Glazer, A., Lindenbaoum, M., and Markovitch, S. (2012), “Learning High-Density Regions for a Generalized Kolmogorov-Smirnov Test in High-Dimensional Data,” Advances in Neural Information Processing Systems, 1, 728–736.
   Google Scholar
• Hartigan, J. A. (1985), “A Failure of Likelihood Asymptotics for Normal Mixtures,” Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (Vol. 2), eds. L. M. Le Camand and R. A. Olshen, pp. 807–810, Belmont, CA: Wadsworth.
   Google Scholar
• Haynes, M. A., MacGillivray, H. L., and Mengersen, K. L. (1997), “Robustness of Ranking and Selection Rules using Generalized g-and-k Distributions,” Journal of Statistical Planning and Inference, 65, 45–66. DOI: 10.1016/S0378-3758(97)00050-5.
   Web of Science ®Google Scholar
• Hyvärinen, A., and Morioka, H. (2016), “Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA,” in Advances in Neural Information Processing Systems, pp. 3765–3773.
   Google Scholar
• Hyvärinen, A., Sasaki, H., and Turner, R. E. (2018), “Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning,” Arxiv Preprint Arxiv:1805.08651.
   Google Scholar
• LeCam, L. M. (1973), “Convergence of Estimates Under Dimensionality Restrictions,” Annals of Statistics, 1, 38–53.
   Web of Science ®Google Scholar
• LeCam, L. M., and Yang, G. L. (1990), Asymptotics in Statistics. Some Basic Concepts, New York: Springer.
   Google Scholar
• Peacock, J. A. (1983), “Two-Dimensional Goodness-of-Fit Testing in Astronomy,” Monthly Notices Royal Astronomy Society, 202, 615–627. DOI: 10.1093/mnras/202.3.615.
   Web of Science ®Google Scholar
• Polonik, W. (1999), “Concentration and Goodness-of-Fit in Higher Dimensions: (Asymptotically) Distribution-Free Methods,” Annals of Statistics, 27, 1210–1229.
   Web of Science ®Google Scholar
• Ramberg, J. S., Tadikamalla, P. R., Dudewicz, E. J., and Mykytka, E. F. (1979), “A Probability Distribution and Its Uses in Fitting Data,” Technometrics, 21, 201–214. DOI: 10.1080/00401706.1979.10489750.
   Web of Science ®Google Scholar
• Ran, Z.-Y., and Hu, B.-G. (2014), “Determining Parameter Identifiability from the Optimization Theory Framework: A Kullback-Leibler Divergence Approach,” Neurocomputing, 142, 307–317. DOI: 10.1016/j.neucom.2014.03.055.
   Web of Science ®Google Scholar
• Ran, Z.-Y., and Hu, B.-G. (2017), “Parameter Identifiability in Statistical Machine Learning: A Review,” Neural Computation, 29, 1151–1203.
   PubMed Web of Science ®Google Scholar
• Rayner, G. D., and MacGillivray, H. L. (2002), “Numerical Maximum Likelihood Estimation for the g-and-k and Generalized g-and-h Distributions,” Statistics and Computing, 12, 57–75.
   Web of Science ®Google Scholar
• Roeder, G., Metz, L., and Kingma, D. P. (2021), “On Linear Identifiability of Learned Representations,” in Proceedings of the 38th International Conference on Machine Learning, PMLR 139. arXiv:2007.00810v3 [stat.ML] 8 Jul 2020
   Google Scholar
• Rothenberg, T. J. (1971), “Identification in Parametric Models,” Econometrica, 39, 577–591. DOI: 10.2307/1913267.
   Web of Science ®Google Scholar
• Sackrowitz, H., and Samuel-Cahn, E. (1999), “p-Values as Random Variables-Expected p-Values,” American Statistician, 53, 326–331. DOI: 10.2307/2686051.
   Web of Science ®Google Scholar
• Stein, C. (1964), “Inadmissibility of the Usual Estimator for the Variance of a Normal Distribution with Unknown Mean,” Annals of the Institute of Statistical Mathematics, 16, 155–160. DOI: 10.1007/BF02868569.
   Web of Science ®Google Scholar
• Tukey, J. W. (1962), “The Future of Data Analysis,” Annals of Mathematical Statistics, 33, 1–67. DOI: 10.1214/aoms/1177704711.
   Google Scholar
• Tukey, J. W. (1977), “Modern Techniques in Data Analysis,” NSF-sponsored Regional Research Conference at Southeastern Massachusetts University, North Dartmouth, MA.
   Google Scholar
• Veres, S.(1987), “Asymptotic Distributions of Likelihood Ratios for Overparameterized ARMA Processes,” Journal of Time Series Analysis, 8, 345–357. DOI: 10.1111/j.1467-9892.1987.tb00446.x.
   Google Scholar
• Watanabe, S. (2001), “Algebraic Analysis of Nonidentifiable Learning Machines,” Neural Computation, 13, 899–933. DOI: 10.1162/089976601300014402.
   PubMed Web of Science ®Google Scholar
• Yan, Y., and Genton, M. G. (2019), “The Tukey g-and-h Distribution,” Significance, 2019, 12–13. DOI: 10.1111/j.1740-9713.2019.01273.x.
   Google Scholar
• Yatracos, Y. G. (2020), “Learning with Matching in Data-Generating Experiments,” DOI: 10.13140/RG.2.2.30964.58245.
   Google Scholar
• Yatracos, Y. G. (2021), “Fiducial Matching for the Approximate Posterior: F-ABC.” DOI: 10.13140/RG.2.2.20775.06568.
   Google Scholar
• Yatracos, Y. G. (2022), “Limitations of the Wasserstein MDE for Univariate Data,” Statistics and Computing, 32, 95. DOI: 10.1007/s11222-022-10146-7.
   Web of Science ®Google Scholar