PreparedLLM: effective <i>pre</i> -pretraining framework for domain-specific large language models-Reference-Cited by-同舟云学术

PreparedLLM: effective pre -pretraining framework for domain-specific large language models

Published:2024-09-08 Issue: Volume: Page:1-24
ISSN:2096-4471
Container-title:Big Earth Data
language:en
Short-container-title:Big Earth Data

Author:

Chen Zhou¹,Lin Ming¹,Wang Zimeng¹,Zang Mingrun¹,Bai Yuqi¹

Affiliation:

1. Department of Earth System Science, Institute for Global Change Studies, Ministry of Education Ecological Field Station for East Asian Migratory Birds, Tsinghua University, Beijing, China

Funder

Major Program of the National Natural Science Foundation of China

Publisher

Informa UK Limited

Link

https://www.tandfonline.com/doi/pdf/10.1080/20964471.2024.2396159

Reference57 articles.

1. Allen Institute for AI. (n.d). Introduction: Benchmarking Cross-Task Generalization. Allen Institute for AI. https://instructions.apps.allenai.org/

2. Antoun, W., Baly, F., & Hajj, H. (2020, May). AraBERT: Transformer-based Model for Arabic language understanding. Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, Marseille, France, European Language Resource Association (pp. 9–15). https://aclanthology.org/2020.osact-1.2

3. Artificial intelligence: revolutionizing cardiology with large language models

4. Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., & Erlingsson, U. (2021, August). Extracting training data from large language models. Proceedings of the 30th USENIX Security Symposium, USENIX Association (pp. 2633–2650). https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting

5. Towards using Few-Shot Prompt Learning for Automating Model Completion