Краткий обзор типологии нейронов и анализ использования мемристорных кроссбаров

Обложка

Цитировать

Полный текст

Открытый доступ Открытый доступ
Доступ закрыт Доступ предоставлен
Доступ закрыт Только для подписчиков

Аннотация

Нейроморфные технологии, использующие искусственные нейроны и синапсы, могут предложить более эффективное решение для исполнения алгоритмов искусственного интеллекта, чем традиционные вычислительные системы. Недавно были разработаны искусственные нейроны, использующие мемристоры, однако они имеют ограниченную биологическую динамику и не могут взаимодействовать напрямую с искусственными синапсами в интегрированной системе. Целью работы является обзор уровней сложности и функций нейронов и синапсов, а также анализ схемотехнического воплощения отдельных типов нейронов и нейронных сетей.

Полный текст

Доступ закрыт

Об авторах

А. А. Токарев

Российский технологический университет (РТУ МИРЭА)

Автор, ответственный за переписку.
Email: santokar5@gmail.com
Россия, Москва

И. А. Хорин

Физико-технологический институт им. К.А. Валиева РАН

Email: khorin@ftian.ru
Россия, Москва

Список литературы

  1. Strukov D.B., Snider G.S., Stewart D.R., Williams R.S. The missing memristor found // Nature, 2008. V. 453. P. 80–83.
  2. Roy K., Laiswal A., Panda P. Towards spike-based machine intelligence with neuromorphic computing // Nature, 2019. V. 575. P. 607—617.
  3. Jeon W.K.G., Lee J., Lee H. et al. Deep learning with GPUs. In S. Kim, & G. C. Deka (Eds.), Hardware Accelerator Systems for Artificial Intelligence and Machine Learning (2021). (pp. 167—215). (Advances in Computers; Vol. 122). Academic Press Inc.
  4. Capra M., Bussolino B., Marchisio A. et al. Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead // IEEE Access, 2020. V. 8. P. 225134-225180.
  5. Sebastian A., Le Gallo M., Khaddam-Aljameh R. et al. Memory devices and applications for in-memory computing // Nat. Nanotechnol., 2020. V. 15. P. 529–544.
  6. Xia Q.-F., Yang J.-J. Memristive crossbar arrays for brain-inspired computing // Nat. Mater., 2019. V. 18(4). P. 309–323.
  7. Jo S.H., Chang T., Ebong I. et al. Nanoscale memristor device as synapse in neuromorphic systems // Nano Lett., 2010. V. 10. P. 1297–1301.
  8. Yu S., Wu Y., Jeyasingh R. et al. An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation // IEEE Trans. Electron Devices, 2011, V. 58. P. 2729–2737.
  9. Ohno T., Hasegawa T., Tsuruoka T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses // Nat. Mater., 2011. V. 10. P. 591–595.
  10. Pershin Y.V., Di Ventra M. Neuromorphic, digital, and quantum computation with memory circuit elements // Proc. IEEE, 2012. V. 100. P. 2071–2080.
  11. Hu S.G., Liu Y., Liu Z. et al. Associative memory realized by a reconfgurable memristive Hopfeld neural network // Nat. Commun., 2015. V. 6. 7522 (2015).
  12. Burr G.W., Shelby R.M., Sidler S. et al. Experimental demonstration and tolerancing of a large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element // 2014 IEEE International Electron Devices Meeting, pp. 29.5.1–29.5.4.
  13. Wong H.−S.P., Lee H.−Y., Yu S. et al. Metal−oxide RRAM // Proceedings of the IEEE, 2012. V. 100. № 6. P. 1951—1970.
  14. Yang J.J., Strukov D.B., Stewart D.R. Memristive devices for computing // Nature Nanotechnology, 2013. V. 8. № 1. P. 13—24.
  15. van de Burgt Y., Lubberman E., Fuller E.J. et al. A non-volatile organic electrochemical device as a low-voltage artifcial synapse for neuromorphic computing // Nat. Mater., 2017. V. 16(4). P. 414–418.
  16. Merolla P.A., Arthur J.V., Alvarez-Icaza R. et al. A million spiking-neuron integrated circuit with a scalable communication network and interface // Science, 2014. V. 345. P. 668–673.
  17. Sourikopoulos I., Hedayat S., Loyez C. et al. A 4-fJ/spike artifcial neuron in 65 nm CMOS technology // Front. Neurosci., 2017. V. 11. 123.
  18. Li C., Belkin D., Li Y. et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks // Nat. Commun., 2018. V. 9. 2385.
  19. Trepel M. Neuranatomie: Struktur und Funktion 7. Auflage, Elsevier. Munchen. 2012. 439 P.
  20. Mead C. Neuromorphic electronic systems // Proc. IEEE, 1990. V. 78. P. 1629–1636.
  21. Kheradpisheh S.R., Ganjtabesh M., Thorpe S.J. et al. STDP-based spiking deep convolutional neural networks for object recognition // Neural Networks, 2018. V. 99. P. 56–67.
  22. Mozafari M., Ganjtabesh M., Nowzari-Daliniet A. al. Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks // Pattern Recognition, 2019. V. 94. P. 87—95.
  23. Peng H., Gan L., Guo X. Memristor based Spiking Neural Networks: Cooperative Development of Neural Network Architecture/Algorithms and Memristors // Chip, 2024. V. 3(2). 100093.
  24. Ianov D., Chezhegov A., Kiselev M. et al. Neuromorphic artificial intelligence systems // Frontiers in Neuroscience, 2022. doi: 10.3389/fnins.2022.959626
  25. Chen A., Datta S., Hu X.S. et al. A survey on architecture advances enabled by emerging beyond-CMOS technologies // IEEE Design & Test, 2019. V. 36. P. 46–68.
  26. Dragoman M., Dragoman D. Atomic-Scale Electronics Beyond CMOS. Springer Nature Switzerland AG. 2021.
  27. Hebb D.O. The Organization of Behavior: A Neuropsychological Theory. John Wiley & Sons, Inc. 1949.
  28. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain // Psychol. Rev., 1958, V. 65. 386.
  29. Ивахненко А.Г., Лапа В.Г. Кибернетические предсказывающие устройства. Киев – 1965. 219 с.
  30. Werbos P.J. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Harvard Univ. Cambridge, Massachusetts. 1974. 453 p.
  31. Kelley H.J. Gradient theory of optimal flight paths // ARS J., 1960. V. 30. P. 947–954.
  32. Rumelhart D.E., Hinton G.E., Williams R.J. Learning representations by back-propagating errors // Nature, 1986. V. 323. P. 533–536.
  33. Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities // Proc. Natl Acad. Sci. USA, 1982. V. 79. P. 2554–2558.
  34. Kohonen T. Self-organized formation of topologically correct feature maps // Biol. Cybern., 1982. V. 43. P. 59–69.
  35. Manipatrun, S., Nikonov D.E., Young I.A. Beyond CMOS computing with spin and polarization // Nat. Phys., 2018. V. 14. P. 338–343.
  36. Preskill J. Quantum computing in the NISQ era and beyond // Quantum, 2018. V. 2. P. 79.
  37. Solli D.R., Jalali B. Analog optical computing // Nat. Photonics, 2015. V. 9. P. 704–706.
  38. Паун Г., Розенберг Г., Саломаа А. ДНК-компьютер. Новая парадигма вычислений: пер. с англ. – М.: Мир, 2003. – 528 с.
  39. Mead C. Neuromorphic electronic systems // Proc. IEEE. 1990. V. 78. P. 1629–1636.
  40. Kephart J.O., Chess D.M. The vision of autonomic computing // Computer, 2003. V. 36. P. 41–50.
  41. Kish L.B. Thermal noise driven computing // Appl. Phys. Lett., 2006. V. 89. 144104.
  42. Unger R., Moult J. Towards computing with proteins // Proteins, 2006. V. 63. P. 53–64.
  43. Upadhyay N.K., Jiang H., Wang Z. et al. Emerging memory devices for neuromorphic computing // Adv. Mater. Technol., 2019. V. 4. 1800589.
  44. Bengio Y. Deep Learning of Representations for Unsupervised and Transfer Learning // JMLR: Workshop and Conference Proceedings, 2012, V. 27. P. 17—37.
  45. Schmidhuber J. Deep learning in neural networks: An overview // Neural Netw., 2015. V. 61 P. 85–117.
  46. Chua L.O. Local activity is the origin of complexity // Int. J. Bifurc. Chaos, 2005. V. 15. P. 3435–3456.
  47. Csete M.E., Doyle J.C. Reverse engineering of biological complexity // Science, 2002. V. 295. P. 1664–1669.
  48. Stanley H.E., Amaral L.A., Buldyrev S.V. et al. Self-organized complexity in economics and finance // Proc. Natl Acad. Sci. USA, 2002. V. 99. P. 2561–2565.
  49. Arthur W.B. Foundations of complexity economics // Nat. Rev. Phys., 2021. V. 3. P. 136–145.
  50. Wilson H.R. Voluntary generation of hyperchaotic visuo-motor patterns // Sci. Rep., 2019. V. 9. 13819.
  51. Yao P., Wu H., Gao B., et al. Fully hardware-implemented memristor convolutional neural network // Nature, 2020. V. 577 P. 641—646.
  52. Hodgkin A.L., Huxley A.F. A quantitative description of membrane current and its application to conduction and excitation in nerve // J. Physiol., 1952. V. 117(4). P. 500–544.
  53. Kistler W.G.W.M. Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press. 2002. 498 p.
  54. Izhikevich E.M. Simple model of spiking neurons // IEEE Trans. Neural Netw., 2003. V. 14. P. 1569–1572.
  55. Segee B. Methods in Neuronal Modeling: from Ions to Networks, 2nd Edition // Comput. Sci. Eng., 1999. V. 1(1). P. 81—81.
  56. Taherkhani A., Belatreche A., Liet Y. al. A review of learning in biologically plausible spiking neural networks // Neural Netw., 2020. V. 122. P. 253–272.
  57. Fang W., Yu Z., Chen Y. et al. Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks // 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, QC, Canada, 2021. P. 2641—2651.
  58. Duan Q., Jing Z., Zou X. et al. Spiking neurons with spatiotemporal dynamics and gain modulation for monolithically integrated memristive neural networks // Nat. Commun., 2020. V. 11. 3399.
  59. Wang Y.-H., Gong T.-C., Ding Y.-X. et al. Redox memristors with volatile threshold switching behavior for neuromorphic computing // Journal of Electronic Science and Technology, 2022. V. 20. 100177.
  60. Wang Z.-R., Joshi S., Savel’ev S. et al. Fully memristive neural networks for pattern classification with unsupervised learning // Nat. Electron., 2018. V. 1(2). P. 137–145.
  61. Kim Y., Kwon Y.J., Kwon D.E., Yoon K.J. Nociceptive Memristor // Adv. Mater., 2018. V. 30, 1704320.
  62. Davies M., Srinivasa N., Lin T.-H., et al. Loihi: a neuromorphic manycore processor with on-chip learning / IEEE Micro, 2018. V. 38. №1. P. 82—99.
  63. Imam N., Cleland T.A. Rapid online learning and robust recall in a neuromorphic olfactory circuit // Nat. Mach. Intell., 2020. V. 2(3). P. 181–191.
  64. Merolla P.A., Arthur J.V., Alvarez-Icaza R., et al. A million spiking-neuron integrated circuit with a scalable communication network and interface // Science, 2014. V. 345(6197). P. 668–673.
  65. DeBole M.V., Taba B., Amir A., et al. TrueNorth: Accelerating from zero to 64 million neurons in 10 years // Computer, 2019. V. 52(5). P. 20–29.
  66. Benjamin B.V., Gao P.-R., Mcquinn E. et al. Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations // Proc. IEEE, 2014. V. 102(5). P. 699–716.
  67. Beck M.E., Shylendra A., Sangwan V.K., et al. Spiking neurons from tunable Gaussian heterojunction transistors // Nat. Commun., 2020. V. 11(1). P. 1565, 1–8.
  68. Covi E., Donati E., Liang X.-P., et al. Adaptive extreme edge computing for wearable devices // Front. Neurosci., 2021. V. 15. P. 611300, 1—27.
  69. Pickett M.D., Medeiros-Ribeiro G., Williams R.S. A scalable neuristor built with Mott memristors // Nat. Mater., 2013. V. 12(2). P. 114–117.
  70. Yi W., Tsang K.K., Lam S.K., et al. Biological plausibility and stochasticity in scalable VO2 active memristor neurons // Nat. Commun., 2018. V. 9 P. 4661, 1–10.
  71. Gao L.-G., Chen P.-Y., Yu S.-M. NbOx based oscillation neuron for neuromorphic computing // Appl. Phys. Lett., 2017. V. 111(10). P. 103503, 1–4.
  72. Kumar S., Williams R.S., Wang Z.-W. Third-order nanocircuit elements for neuromorphic engineering // Nature, 2020. V. 585. 7826. P. 518–523.
  73. Zhang X.-M., Wang W., Liu Q., et al. An artificial neuron based on a threshold switching memristor // IEEE Electron. Device Lett., 2018. V. 39(2). P. 308–311.
  74. Zhang X.-M., Lu J., Wang Z.-R., et al. Hybrid memristor-CMOS neurons for in-situ learning in fully hardware memristive spiking neural networks // Sci. Bull., 2021. V. 66(16). P. 1624–1633.
  75. Fu T.-D., Liu X.-M., Gao H.-Y., et al. Bioinspired bio-voltage memristors // Nat. Commun., 2020. V. 11(1). P. 1861, 1—10.
  76. Li X.-Y., Tang J.-S., Zhang Q.-T., et al. Power-efficient neural network with artificial dendrites // Nat. Nanotechnol., 2020. V. 15(9) P. 776–782.
  77. Wu W., Wu H., Gao B. et. al. A methodology to improve linearity of analog RRAM for neuromorphic computing // 2018 IEEE Symposium on VLSI Technology, Honolulu, HI, USA, 2018. P. 103–104.
  78. Kim T., Hu S., Kim J. et al. Spiking Neural Network (SNN) With Memristor Synapses Having Non-linear Weight Update // Front. Comput. Neurosci., 2021. V. 15. 646125.
  79. Rao M., Tang H., Wu J. et al. Thousands of conductance levels in memristors integrated on CMOS // Nature, 2023. V. 615. P. 823—829.
  80. Liang S., Yin S., Liu L. et al. FP-BNN: Binarized neural network on FPGA // Neurocomputing, 2018. V. 275. P. 1072—1086.
  81. Simons T., Lee D.-J. A Review of Binarized Neural Networks // Electronics, 2019. V. 8. P. 661.
  82. Zhang W., Yao P., Gao B. et al. Edge learning using a fully integrated neuro-inspired memristor chip // Science, 2023. V. 381. P. 1205—1211.
  83. Balaji A., Das A., Wu Y. et al. Mapping Spiking Neural Networks to Neuromorphic Hardware // IEEE Trans. Very Large Scale Integr. (VLSI) Syst., 2020. V. 28. P. 76—86.
  84. Yadav D.N., Thangkhiew P.L., Chakraborty S., et al. Efficient grouping approach for fault tolerant weight mapping in memristive crossbar array // Memories — Materials, Devices, Circuits and Systems, 2023. V. 4. 100045.
  85. Zhang B., Uysal N., Fan D. et al. Handling Stuck-at-Fault Defects Using Matrix Transformation for Robust Inference of DNNs // IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020. V. 39. №10. P. 2448—2460.
  86. Xu Y., Jin S., Wang Y., Qi Y. Aggressive Fault Tolerance for Memristor Crossbar-Based Neural Network Accelerators by Operational Unit Level Weight Mapping // IEEE Access, 2021. V. 9. P. 102828—102834.
  87. Xia L., Huangfu W., Tang T., et al. Stuck-at Fault Tolerance in RRAM Computing Systems // IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2018. V. 8 №1. P. 102—115.
  88. Li J., Ma L., Sham C.-W., Fu C. A Novel Computing Paradigm for MobileNetV3 using Memristor // arXiv:2402.10512v1 [cs.AR]. 16 Feb 2024. 1—15.
  89. Yan B.-N., Chen Y.-R., Li H. Challenges of memristor based neuromorphic computing system // Sci. China Inf. Sci., 2018. V. 61(6). P. 060425, 1—3.
  90. Sung C., Hwang H., Yoo I.K. Perspective: a review on memristive hardware for neuromorphic computation // J. Appl. Phys., 2018. V. 124(15). P. 151903, 1—13.

Дополнительные файлы

Доп. файлы
Действие
1. JATS XML
Скачать
2. Рис. 1. Примеры синаптического поведения, требующие различных порядков сложности.

Скачать (233KB)
Метаданные
3. Рис. 2. Примеры нейронного поведения, требующие различных порядков сложности.

Скачать (222KB)
Метаданные
4. Рис. 3. Принципиальные схемы (а) модели нейронной цепи Ходжкина–Хаксли и (б) модели нейронной цепи LIF [59].

Скачать (117KB)
Метаданные
5. Рис. 4. Схема нейристора, построенного с использованием двух наноразмерных мемристоров в соответствии с моделью Ходжкина—Хаксли [69].

Скачать (102KB)
Метаданные
6. Рис. 5. Схематическая иллюстрация схемы нейрона [76].

Скачать (74KB)
Метаданные
7. Рис. 6. Универсальная вычислительная архитектура нейронных сетей на основе мемристоров [51].

Скачать (444KB)
Метаданные

© Российская академия наук, 2024