A simple differentiable programming language-Reference-Cited by-同舟云学术

A simple differentiable programming language

Published:2019-12-20 Issue:POPL Volume:4 Page:1-28
ISSN:2475-1421
Container-title:Proceedings of the ACM on Programming Languages
language:en
Short-container-title:Proc. ACM Program. Lang.

Author:

Abadi Martín¹,Plotkin Gordon D.¹

Affiliation:

1. Google Research, USA

Abstract

Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the mathematical notion of derivative is not always entirely clear---discrepancies can arise, sometimes inadvertently. In order to study automatic differentiation in such programming contexts, we define a small but expressive programming language that includes a construct for reverse-mode differentiation. We give operational and denotational semantics for this language. The operational semantics employs popular implementation techniques, while the denotational semantics employs notions of differentiation familiar from real analysis. We establish that these semantics coincide.

Publisher

Association for Computing Machinery (ACM)

Link

https://dl.acm.org/doi/pdf/10.1145/3371106

Reference45 articles.

1. Deep Learning with Differential Privacy

2. Samson Abramsky and Achim Jung . 1994. Domain theory . In Handbook of Logic in Computer Science (Vol. 3), Samson Abramsky, Dov M . Gabbay, and T. S. E. Maibaum (Eds.). Oxford University Press, Inc. , 1–168. Samson Abramsky and Achim Jung. 1994. Domain theory. In Handbook of Logic in Computer Science (Vol. 3), Samson Abramsky, Dov M. Gabbay, and T. S. E. Maibaum (Eds.). Oxford University Press, Inc., 1–168.

3. Akshay Agrawal , Akshay Naresh Modi , Alexandre Passos, Allen Lavoie, Ashish Agarwal, Asim Shankar, Igor Ganichev, Josh Levenberg, Mingsheng Hong, Rajat Monga, et al. 2019 . TensorFlow Eager: A multi-stage, Python-embedded DSL for machine learning. arXiv preprint arXiv:1903.01855 (2019). Akshay Agrawal, Akshay Naresh Modi, Alexandre Passos, Allen Lavoie, Ashish Agarwal, Asim Shankar, Igor Ganichev, Josh Levenberg, Mingsheng Hong, Rajat Monga, et al. 2019. TensorFlow Eager: A multi-stage, Python-embedded DSL for machine learning. arXiv preprint arXiv:1903.01855 (2019).

4. Shun-ichi Amari. 1996. Neural learning in structured parameter spaces — natural Riemannian gradient . In Advances in Neural Information Processing Systems 9, NIPS , M. Mozer, M. I. Jordan, and T. Petsche (Eds.). MIT Press , 127–133. Shun-ichi Amari. 1996. Neural learning in structured parameter spaces — natural Riemannian gradient. In Advances in Neural Information Processing Systems 9, NIPS, M. Mozer, M. I. Jordan, and T. Petsche (Eds.). MIT Press, 127–133.

5. Automatic differentiation in machine learning: a survey;Baydin Atilim Günes;Journal of Machine Learning Research,2018

Cited by 30 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Handling the Selection Monad;Proceedings of the ACM on Programming Languages;2025-06-10

2. Distributed Strategies—Past and Future;ACM SIGLOG News;2025-04

3. Scimitar: Functional Programs as Optimization Problems;Proceedings of the 2024 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software;2024-10-17

4. Automatic differentiation for ML-family languages: Correctness via logical relations;Mathematical Structures in Computer Science;2024-09

5. δ is for Dialectica;Proceedings of the 39th Annual ACM/IEEE Symposium on Logic in Computer Science;2024-07-08