Lightweight Domain-Specific Language Models for Cloud Resource Scheduling
DOI:
https://doi.org/10.66421/int.j.datasci.intell.syst61Keywords:
Cloud computing, resource scheduling, small language models, knowledge distillation, constrained decoding, integer programmingAbstract
Cloud schedulers place dependency-constrained tasks on heterogeneous machines while balancing completion time, utilisation and service-level objectives. Recent work shows that a large language model supplies useful scheduling intuition once its proposals are repaired by a mathematical solver, yet a foundation-model call costs hundreds of milliseconds and its price grows with the amount of cluster state that has to be described, which confines the reasoning component to a small fraction of decision epochs. This paper asks whether the reasoning component itself can be made small enough to run on every decision. We present SchedLM, a 0.36 M-parameter causal language model trained from scratch over a scheduling-specific token language. Three ingredients make a model of this size sufficient. A node-class abstraction keeps the prompt at 143 tokens whatever the cluster size. Distillation from a per-epoch integer-programming teacher supervises a defer-free action space, so that deferral is produced by the constraints rather than learned as an action. Feasibility-constrained decoding masks infeasible node classes at every decoding step, so the model emits only executable schedules. On trace-calibrated DAG workloads SchedLM retains 96.2–98.4% of the teacher's weighted job completion time and improves on the Kubernetes default scheduler by 7.1–8.5% in completion time, by 12.9–25.3% in SLA violations and by up to 14.9% in makespan, with zero infeasible assignments. Median decision latency grows 1.20
References
[1] Xing, S., Wang, Y., & Liu, W. (2025). Self-adapting CPU scheduling for mixed database workloads via hierarchical deep reinforcement learning. Symmetry, 17(7), 1109.
[2] Ji, E., Wang, Y., Xing, S., & Jin, J. (2025). Hierarchical reinforcement learning for energy-efficient API traffic optimization in large-scale advertising systems. IEEE Access.
[3] Chen, T., Shang, W., & Guo, Z. (2024). Xsla: An SLA-aware heterogeneous XPU scheduling framework for edge AI inference. World Journal of Information Technology, 2(2), 48–59.
[4] Zou, J., Qin, Y., & Rhee, M. (2024). Beyond model scale: A task-aware reliability benchmark for large language models in clinical decision support. Innovation and Technology Studies, 1(1), 39–47.
[5] Fang, Q. (2024). Bias-aware large language models for equitable educational feedback and assessment. Trends in Social Sciences and Humanities Research.
[6] V. K. Vavilapalli et al., “Apache Hadoop YARN: Yet another resource negotiator,” in Proc. ACM SoCC, 2013, pp. 1–16.
[7] H. Mao, M. Schwarzkopf, S. B. Venkatakrishnan, Z. Meng, and M. Alizadeh, “Learning scheduling algorithms for data processing clusters,” in Proc. ACM SIGCOMM, 2019, pp. 270–288.
[8] M. Schwarzkopf, A. Konwinski, M. Abd-El-Malek, and J. Wilkes, “Omega: Flexible, scalable schedulers for large compute clusters,” in Proc. EuroSys, 2013, pp. 351–364.
[9] A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and I. Stoica, “Dominant resource fairness: Fair allocation of multiple resource types,” in Proc. USENIX NSDI, 2011, pp. 323–336.
[10] R. Grandl, G. Ananthanarayanan, S. Kandula, S. Rao, and A. Akella, “Multi-resource packing for cluster schedulers,” in Proc. ACM SIGCOMM, 2014, pp. 455–466.
[11] I. Gog, M. Schwarzkopf, A. Gleave, R. N. M. Watson, and S. Hand, “Firmament: Fast, centralized cluster scheduling at scale,” in Proc. USENIX OSDI, 2016, pp. 99–115.
[12] C. Delimitrou and C. Kozyrakis, “Quasar: Resource-efficient and QoS-aware cluster management,” in Proc. ACM ASPLOS, 2014, pp. 127–144.
[13] H. Topcuoglu, S. Hariri, and M.-Y. Wu, “Performance-effective and low-complexity task scheduling for heterogeneous computing,” IEEE Trans. Parallel Distrib. Syst., vol. 13, no. 3, pp. 260–274, 2002.
[14] E. Cortez, A. Bonde, A. Muzio, M. Russinovich, M. Fontoura, and R. Bianchini, “Resource Central: Understanding and predicting workloads for improved resource management in large cloud platforms,” in Proc. ACM SOSP, 2017, pp. 153–167.
[15] R. Bhardwaj et al., “Cilantro: Performance-aware resource allocation for general objectives via online feedback,” in Proc. USENIX OSDI, 2023, pp. 623–643.
[16] Y. Bengio, A. Lodi, and A. Prouvost, “Machine learning for combinatorial optimization: A methodological tour d’horizon,” Eur. J. Oper. Res., vol. 290, no. 2, pp. 405–421, 2021.
[17] I. Trummer, “DB-BERT: A database tuning tool that ‘reads the manual’,” in Proc. ACM SIGMOD, 2022, pp. 190–203.
[18] S. Lai, Z. Xu, W. Zhang, H. Liu, and H. Xiong, “LLMLight: Large language models as traffic signal control agents,” 2023, arXiv:2312.16044.
[19] S. Kambhampati et al., “Position: LLMs can’t plan, but can help planning in LLM-Modulo frameworks,” in Proc. Int. Conf. Mach. Learn., 2024.
[20] K. Valmeekam, M. Marquez, S. Sreedharan, and S. Kambhampati, “On the planning abilities of large language models: A critical investigation,” in Proc. Adv. Neural Inf. Process. Syst., 2023, pp. 75993–76005.
[21] G. Ding, S. Yang, H. Lin, Z. Chen, and J. S. Yang, “LLM-driven adaptive cloud resource scheduling: Bridging reasoning intelligence with optimization guarantees,” IEEE Open J. Comput. Soc., vol. 7, pp. 560–573, 2026.
[22] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” 2015, arXiv:1503.02531.
[23] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,” 2019, arXiv:1910.01108.
[24] X. Jiao et al., “TinyBERT: Distilling BERT for natural language understanding,” in Findings of EMNLP, 2020, pp. 4163–4174.
[25] S. Gunasekar et al., “Textbooks are all you need,” 2023, arXiv:2306.11644.
[26] P. Zhang, G. Zeng, T. Wang, and W. Lu, “TinyLlama: An open-source small language model,” 2024, arXiv:2401.02385.
[27] E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “GPTQ: Accurate post-training quantization for generative pre-trained transformers,” in Proc. Int. Conf. Learn. Represent., 2023.
[28] T. Scholak, N. Schucher, and D. Bahdanau, “PICARD: Parsing incrementally for constrained auto-regressive decoding from language models,” in Proc. EMNLP, 2021, pp. 9895–9901.
[29] S. Geng, M. Josifoski, M. Peyrard, and R. West, “Grammar-constrained decoding for structured NLP tasks without finetuning,” in Proc. EMNLP, 2023, pp. 10932–10952.
[30] J. D. Ullman, “NP-complete scheduling problems,” J. Comput. Syst. Sci., vol. 10, no. 3, pp. 384–393, 1975.
[31] C. Reiss, J. Wilkes, and J. L. Hellerstein, “Google cluster-usage traces: Format + schema,” Google Inc., White Paper, 2011.
[32] J. Cao, A. Ganesh, J. Cai, R. Southwell, E. M. Perkoff, M. Regan, K. Kann, J. H. Martin, M. Palmer, and S. D'Mello, “A comparative analysis of automatic speech recognition errors in small group classroom discourse,” in Proc. 31st ACM Conf. User Modeling, Adaptation and Personalization (UMAP), 2023, pp. 250–261.
[33] Qin, Y., Zou, J., & Rhee, M. (2022). An AI-driven intelligent transaction execution framework for internet finance platforms: Cost-sensitive real-time integrity interdiction. Eurasia Journal of Science and Technology, 4(2), 36–44.
[34] Wang, Y., & Zhang, C. (2023). AERIS: Adaptive executor scoped resource management for privacy preserving data processing in Apache Spark. Eurasia Journal of Science and Technology, 5(2), 25-33.
[35] Zhao, W., & Wang, B. (2022). CAPA: A prediction‑driven autoscaling framework for SLO‑aware cloud‑native machine learning inference. Journal of Computer Science and Electrical Engineering, 4(1), 8-16.
[36] Guo, Z., Chen, T., & Shang, W. (2022). Class‑conditional intermediate‑domain adversarial adaptation for cross‑device acoustic scene classification. Journal of Computer Science and Electrical Engineering, 4(1), 26-35
[37] Shen, A., & Huang, S. (2022). A systems-aware framework for intelligent large-scale recommendation models. Journal of Computer Science and Electrical Engineering.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Liam Walker, Aisha Rahman, Victor Chen

This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are published under the Creative Commons Attribution 4.0 (CC BY 4.0) License. Users may share, copy, adapt, and distribute the work for any purpose, provided proper credit is given to the original author(s) and source. Authors retain copyright.
