TGRAC: Temporal Graph Residual Actor-Critic for Dynamic Task Scheduling in Heterogeneous Distributed Systems

Authors

  • David Nolan Pennsylvania State University, University Park, PA, USA
  • Claire Huang Pennsylvania State University, University Park, PA, USA

DOI:

https://doi.org/10.66421/int.j.datasci.intell.syst63

Keywords:

heterogeneous computing,, task scheduling, temporal graph learning, reinforcement learning, DAG scheduling, actor-critic

Abstract

Dynamic task scheduling in heterogeneous clusters is difficult because a scheduler must reason simultaneously about workflow dependencies, device heterogeneity, and nonstationary background load. We present TGRAC, a temporal-graph residual actor-critic scheduler that combines (i) a message-passing encoder for directed acyclic task graphs, (ii) a recurrent encoder of recent per-node load histories, and (iii) a masked residual policy built on a short-horizon earliest-finish-time prior. This hybrid design preserves the strong inductive bias of list scheduling while allowing reinforcement learning to correct placements when short-term resource dynamics make a snapshot estimate unreliable. We evaluate the executable implementation on a trace-informed synthetic benchmark whose sparse DAG structure and heterogeneous-resource motivation are grounded in published Alibaba cluster-trace studies. The reported numbers are produced directly by the supplied experiment code: 80 training episodes followed by 50 paired test DAGs in each of three scenarios (normal dynamics, deterministic load spikes, and monotone load drift). TGRAC reduces mean makespan by 4.74% relative to static HEFT under load spikes (paired t-test p=0.0024) and by 5.86% relative to the graph-ablated policy. Across normal, spike, and drift scenarios, graph-aware ordering is consistently important, while temporal history provides its clearest benefit during abrupt load variation. The results support a practical design point: combining a graph-structured scheduling prior with temporal state and a learned residual policy can improve robustness without discarding efficient classical scheduling structure.

 

References

[1] Mai, N. T., Cao, W., & Fang, Q. (2025). A study on how LLMs (eg GPT-4, chatbots) are being integrated to support tutoring, essay feedback and content generation. Journal of Computing and Electronic Information Management, 18(3), 43-52.

[2] Fang, Q., & Liu, W. (2026). Neuro-symbolic pedagogical alignment (NSPA) for long-horizon classroom discourse analysis: Mitigating dialect bias via counterfactual preference optimization. Computers and Education: Artificial Intelligence, 100664.

[3] Wang, Y., & Xing, S. (2025). AI-driven CPU resource management in cloud operating systems. Journal of Computer and Communications, 13(06), 135-149.

[4] R. Grandl, S. Kandula, S. Rao, A. Akella, and J. Kulkarni, “GRAPHENE: Packing and dependency-aware scheduling for data-parallel clusters,” in Proc. USENIX OSDI, 2016, pp. 81-97.

[5] K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica, “Sparrow: Distributed, low latency scheduling,” in Proc. ACM SOSP, 2013, pp. 69-84.

[6] C. Delimitrou and C. Kozyrakis, “Paragon: QoS-aware scheduling for heterogeneous datacenters,” in Proc. ASPLOS, 2013, pp. 77-88, doi: 10.1145/2451116.2451125.

[7] C. Delimitrou and C. Kozyrakis, “Quasar: Resource-efficient and QoS-aware cluster management,” in Proc. ASPLOS, 2014, pp. 127-144, doi: 10.1145/2541940.2541941.

[8] H. Mao, M. Alizadeh, I. Menache, and S. Kandula, “Resource management with deep reinforcement learning,” in Proc. ACM HotNets, 2016.

[9] H. Mao, M. Schwarzkopf, S. B. Venkatakrishnan, Z. Meng, and M. Alizadeh, “Learning scheduling algorithms for data processing clusters,” in Proc. ACM SIGCOMM, 2019, pp. 270-288.

[10] A. Mirhoseini et al., “Device placement optimization with reinforcement learning,” in Proc. ICML, 2017.

[11] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proc. ICLR, 2017.

[12] W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Proc. NeurIPS, 2017.

[13] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in Proc. ICLR, 2018.

[14] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in Proc. ICML, 2017, pp. 1263-1272.

[15] D. Xu, C. Ruan, E. Korpeoglu, S. Kumar, and K. Achan, “Inductive representation learning on temporal graphs,” in Proc. ICLR, 2020.

[16] E. Rossi, B. Chamberlain, F. Frasca, D. Eynard, F. Monti, and M. Bronstein, “Temporal graph networks for deep learning on dynamic graphs,” arXiv:2006.10637, 2020.

[17] S. Yang, G. Ding, Z. Chen, and J. S. Yang, “GART: Graph neural network-based adaptive and robust task scheduler for heterogeneous distributed computing,” IEEE Access, vol. 13, pp. 200196-200216, 2025, doi: 10.1109/ACCESS.2025.3633290.

[18] A. Pareja et al., “EvolveGCN: Evolving graph convolutional networks for dynamic graphs,” in Proc. AAAI, vol. 34, no. 4, pp. 5363-5370, 2020, doi: 10.1609/aaai.v34i04.5984.

[19] A. Sankar, Y. Wu, L. Gou, W. Zhang, and H. Yang, “DySAT: Deep neural representation learning on dynamic graphs via self-attention networks,” in Proc. ACM WSDM, 2020.

[20] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018.

[21] V. Mnih et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529-533, 2015, doi: 10.1038/nature14236.

[22] J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel, “High-dimensional continuous control using generalized advantage estimation,” in Proc. ICLR, 2016.

[23] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv:1707.06347, 2017.

[24] J. Dean and L. A. Barroso, “The tail at scale,” Commun. ACM, vol. 56, no. 2, pp. 74-80, 2013, doi: 10.1145/2408776.2408794.

[25] Fang, Q., & Liu, W. (2025). HARLA-ED: Resolving Information Asymmetry and Enhancing Algorithmic Symmetry in Intelligent Educational Assessment via Hybrid Reinforcement Learning. Symmetry, 18(1), 58.

[26] Xing, S., & Wang, Y. (2025). Cross-modal attention networks for multi-modal anomaly detection in system software. IEEE Open Journal of the Computer Society.

[27] Chen, T., Shang, W., & Guo, Z. (2024). Xsla: An SLA-aware heterogeneous XPU scheduling framework for edge AI inference. World Journal of Information Technology, 2(2), 48–59.

[28] Qin, Y., Zou, J., & Rhee, M. (2022). An AI-driven intelligent transaction execution framework for internet finance platforms: Cost-sensitive real-time integrity interdiction. Eurasia Journal of Science and Technology, 4(2), 36–44.

Downloads

Published

2026-09-29