AI inference routing and cost management
Main Article Content
Abstract
Artificial intelligence (AI) inference routing has become a fundamental capability for delivering scalable, responsive, and cost-efficient AI services across cloud, edge, and hybrid computing environments. As organizations increasingly deploy large language models and other computationally intensive AI applications, selecting optimal execution paths for inference requests is essential to maintaining service quality while controlling operational expenses. This study examines the principles, architectures, and optimization techniques that underpin AI inference routing and cost management. It explores routing strategies based on workload distribution, network awareness, adaptive scheduling, reinforcement learning, and collaborative cloud-edge inference. The discussion further evaluates cost drivers, including computational resource consumption, network bandwidth, storage utilization, energy demand, and model-serving overhead, alongside techniques such as dynamic scaling, request batching, model optimization, and intelligent resource allocation that improve efficiency. Performance is assessed using metrics such as latency, throughput, scalability, resource utilization, and cost per inference request. The study also identifies emerging trends, including agentic AI orchestration, heterogeneous infrastructure management, and 6G-enabled edge intelligence, which support more autonomous and resilient inference ecosystems. The findings demonstrate that intelligent routing combined with proactive cost management enhances system performance, improves infrastructure utilization, and promotes economically sustainable AI deployment while maintaining reliability, responsiveness, and high-quality inference outcomes across distributed computing environments.
Downloads
Download data is not yet available.
Article Details
How to Cite
Prabhu, P. (2026). AI inference routing and cost management. SAMRIDDHI : A Journal of Physical Sciences, Engineering and Technology, 18(03), 64-72. https://doi.org/10.18090/samriddhi.v18i03.04
Section
Research Article
References
1. Yu, S., Goudarzi, M., & Toosi, A. N. (2025). Efficient Routing of Inference Requests across LLM Instances in Cloud-Edge Computing. arXiv preprint arXiv:2507.15553.
2. Wang, Z., Sedlak, B., & Dustdar, S. (2026). Active Inference-Based Adaptive Routing for Heterogeneous Edge AI Services. arXiv preprint arXiv:2604.17373.
3. Chen, X., Yu, H., Ni, W., Niyato, D., Zhang, R., Wang, X., ... & Xu, S. (2026). Networking-Aware Energy Efficiency in Agentic AI Inference: A Survey. arXiv preprint arXiv:2604.07857.
4. Farooq, H., Forgeat, J., Cyras, K., Bothe, S., & Chowdhury, M. M. U. (2026, May). STAR: Smart Network-Aware AI-for-RAN Routing. In IEEE INFOCOM 2026-IEEE Conference on Computer Communications (pp. 1-6). IEEE.
5. Zheng, W., She, J., Zhang, W., Chen, Y., Chen, L., Kundu, S., ... & Yao, H. CLEAR: A Cost-Aware Routing System for Edge-Cloud Language Model Collaborative Inference.
6. Fu, Y., Qin, P., & Jiang, C. (2026). Edge AI Inference in 6G Mobile Networks: From Communication-Efficient Techniques to Task-Oriented Optimization. IEEE Communications Surveys & Tutorials.
7. Zhang, C., Dong, M., & Ota, K. (2021). Employ AI to improve AI services: Q-learning based holistic traffic control for distributed co-inference in deep learning. IEEE Transactions on Services Computing, 15(2), 627-639.
8. Parashar, P. (2026). Edge and Cloud-Based AI Inference for Big Data Processing. In Harnessing AI Inference for Intelligent Decision-Making in Real-Time Dataflows (pp. 1-28). IGI Global Scientific Publishing.
9. Jin, Y., & Yang, Z. (2025, January). Scalability optimization in cloud-based AI inference services: Strategies for real-time load balancing and automated scaling. In Proceedings of the 2025 4th International Conference on Big Data, Information and Computer Network (pp. 266-270).
10. Kodali, S. (2020). Optimizing Supply Chain Network Design with AI: Utilizing Reinforcement Learning and Advanced Analytics for Demand Forecasting, Route Optimization, and Cost Minimization. American Journal of Cognitive Computing and AI Systems, 4, 1-36.
11. Ezeagwuna, D. (2026). AI-Driven Intrusion Detection Systems for Cybersecurity. Journal of Science Technology and Social Transformation, 2(02), 37-51.
12. Mukherjee, C. (2026). AI-Based Detection of Deepfakes and Misinformation on Social Media. Euro Vantage journals of Artificial intelligence, 3(2), 9-30.
13. Mehta, S. (2026). Architecting AI Governance & ‘Agent-ready’data layer: Semantic Governance for Autonomous AI workflows. Journal of Science Technology and Social Transformation, 2(02), 29-36.
14. Ezeagwuna, D. (2026). Digital Transformation and Business Resilience: How Service Now-Powered Automation Improves Organizational Performance, Risk Management, and Customer Satisfaction. Journal of Data Analysis and Critical Management, 2(01), 48-62.
15. Rehan, H., Akram, W., & Khan, M. B. S. (2026, May). Leveraging Gated Recurrent Units for Real-Time Tracking of Student Engagement in Healthcare LMS using LLM Analytics. In 2026 Systems and Information Engineering Design Symposium (SIEDS) (pp. 1-6). IEEE.
16. Mehta, S. (2026). From Reactive cleanup to Real-time observability: Modernizing Enterprise Data Quality for RAG pipelines. Journal of Science Technology and Social Transformation, 2(01), 35-44.
17. Ezeagwuna, D. (2026). Enhancing Organizational Cyber Resilience Through the NIST Cybersecurity Framework. International Journal of Technology, Management and Humanities, 12(01), 124-140.
18. Kola, J. N. (2026, June). Secure Financial Analytics Platforms for Regulatory Compliance and Risk Management. In 2026 IEEE 18th International Conference on Computational Intelligence and Communication Networks (CICN) (pp. 794-801). IEEE.
19. Rehan, H., Zimmerman, Z., & Janjua, J. I. (2026, May). Optimizer-Driven Techniques for Retuning Prompts on LLM Backend with Migration across Models for Healthcare and Cyber Security Operations. In 2026 Systems and Information Engineering Design Symposium (SIEDS) (pp. 1-6). IEEE.
20. Kola, J. N. (2026). Enterprise Business Intelligence and Financial Analytics with a focus on Enterprise Data Architecture and Predictive Risk Integration. Adhyayan: A Journal of Management Sciences, 16(01), 20-28.
21. Ravikumar, V. (2025). Therapeutic Bot: Ethical Concerns in AI therapy for Neurodivergence. J Int Scient Re Rep.
22. Mehta, S. (2025). Enhancing Enterprise Data Governance Maturity Through Cloud Based Data Integration Frameworks in Modern Organizations. International Journal of Technology, Management and Humanities, 11(01), 59-67.
23. Rehan, H., Janjua, J. I., Ali, R. A., & Khan, T. A. (2025, December). AI-Driven DNA Insights and Public Health Data Integration for Enhancing Personal Healthcare in Critical Disease Prevention. In 2025 International Conference on AI-Driven STEM Education and Learning Technologies (AISTEMEDU) (pp. 1-6). IEEE.
24. Ezeagwuna, D. (2025). Artificial Intelligence for IT Service Management: Developing a Framework for ROI Optimization Using Service Now and Machine Learning Technologies. Well Testing Journal, 34(S4), 394-419.
25. Rehan, H., & Janjua, J. I. (2025, October). An Intelligent Data-Driven Framework for Measuring Consumer Experience in the Digital Platforms. In 2025 IEEE International Conference on Computing (ICOCO) (pp. 500-505). IEEE.
26. Mehta, S. (2025). Role of Metadata Management and Data Governance in Achieving Regulatory Compliance in Enterprise Data Ecosystems. International Journal of Technology, Management and Humanities, 11(02), 144-152.
27. Warren, Brandon. (2025). BUSINESS VALUE DRIVEN BY ENTERPRISE ARCHITECTURE TRANSFORMATION BRANDON WARREN. Journal of Tianjin University Science and Technology. 58. 10.5281/zenodo.20280991.
28. Adepoju, S. A., & Adepoju, M. A. (2024). From Portals to Case Graphs: A Reference Architecture and Benchmark for Safety Investigation Operations with Agentic Orchestration.
29. Khan, H. A. (2024). DATA-DRIVEN EPIDEMIOLOGICAL MODELING USING MACHINE LEARNING FOR DISEASE SPREAD FORECASTING AND PUBLIC HEALTH DECISION SUPPORT IN THE UNITED STATES. International Journal of Applied Mathematics, 37(6s), 178-192.
30. Kola, J. N. (2024). Longitudinal Cohort Intelligence for Self-Insured Employer Groups: A Predictive Framework for Healthcare Cost Trajectory Modeling and Proactive Risk Intervention. Journal of Information Systems Engineering & Management, 10.
31. Mehta, S. (2024). Architecting the Hub-and-Spoke Governance Model: Balancing Centralized Guardrails with Federated Domain Agility. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 16(04), 206-215.
32. Ezeagwuna, D. (2024). Enterprise Workflow Automation and Workforce Productivity: Evaluating the Economic Benefits of Service Now Adoption Across Industries. International Journal of Technology, Management and Humanities, 10(04), 299-313.
33. Kola, J. N. (2024). Longitudinal Cohort Intelligence for Self-Insured Employer Groups: A Predictive Framework for Healthcare Cost Trajectory Modeling and Proactive Risk Intervention. Journal of Information Systems Engineering & Management, 10.
34. Das, P. K., Kashem, M. A., Ferdus, Z., & Islam, S. (2019, October). Development and application of a new computerized smell generating system. In 2019 Global Conference for Advancement in Technology (GCAT) (pp. 1-5). IEEE.
35. Ferdus, M. Z., Khan, M. N. I., Islam, S., & Kashem, M. A. (2019, October). VFLT: SQA Model for Cyber Physical System. In 2019 Global Conference for Advancement in Technology (GCAT) (pp. 1-4). IEEE.
36. Thakkar, V. B. (2026). From linear logistics to neural supply chains: Predictive machine learning and the rise of autonomous supply chain intelligence. International Journal of AI, BigData, Computational and Management Studies, 7(1), 153-161.
37. Suseelan, S. (2026). LLM-Based Human-machine Teaming for Air Traffic Controllers. International Journal of Humanities and Information Technology, 8(1), 122-129.
38. Verma, A. (2022). Twin Poisoning: Analyzing the Impact of False Data Injection Attacks on Digital Twin-Based Decision Support Systems. International Journal of Technology, Management and Humanities, 8(03), 39-61.
39. Thakkar, V. B. (2026). A Unified Framework for Integrating Generative AI Across the Agile Software Development Lifecycle (SDLC). International Journal of Humanities and Information Technology, 8(2), 11-18.
40. Suseelan, S. (2026). Multimodal AI for Pilot Skill Assessment Using Physiological Signals. International Journal of Technology, Management and Humanities, 12(02), 74-83.
41. Thakkar, V. B. (2026). Asynchronous Generative AI: Automating Qualitative Data Workflows in Enterprise Systems. International Journal of Science, Technology & Society, 10(01), 1-14.
42. Ferdus, M. Z., Islam, S., & Kashem, M. A. (2019, October). An innovative load balancing cluster composition of wireless sensor networks. In 2019 global conference for advancement in technology (GCAT) (pp. 1-4). IEEE.
43. Islam, S. J., Islam, S., Ferdus, M. Z., Khan, M. N. I., Kashem, M. A., & Islam, M. S. (2020, September). Load compactness and recognizing area aware cluster head selection of wireless sensor networks. In 2020 International conference on computing and information technology (ICCIT-1441) (pp. 1-4). IEEE.
44. Thakkar, V. B. (2025). Algorithmic Trust, Governance, and Integration Bottlenecks of AI Tools in Legacy Project Environments. International Journal of Humanities and Information Technology, 7(01), 80-89.
Mazumder, P. T. (2025). Harnessing fintech innovations for anti-money laundering: a data-driven approach. Available at SSRN 5259084.
45. Suseelan, S. (2025). Big Data Analytics in Air Traffic Management: Enhancing Aviation Safety, Operational Efficiency, and Predictive Decision-Making. Journal of Science Technology and Social Transformation, 1(02), 61-73.
46. Islam, S., Khan, M. N. I., Ferdus, M. Z., Islam, S. J., & Kashem, M. A. (2020, September). Improving throughput using cooperating TDMA scheduling of wireless sensor networks. In 2020 International Conference on Computing and Information Technology (ICCIT-1441) (pp. 1-4). IEEE.
47. Naidu, K. J. (2013). Performance Optimization Of ETL Pipelines In Distributed Data Warehouse Environments: A Network-Aware Scheduling Approach. International Journal of Advance Industrial Engineering, 1(03), 63-67.
48. Ravikumar, V. (2014). Fair and optimal resource allocation in wireless sensor networks.
49. Naidu, K. J. (2014). Secure OLAP Reporting Architectures: Integrating Role-based Access Control and Query Execution Plan Optimization for Enterprise Analytical Environments. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 5(02), 155-159.
50. Kola, J. N. (2011). An Integrated Framework for Data Mining and Distributed Database Optimization in Resource-Constrained Network Environments. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 2(02), 82-86.
2. Wang, Z., Sedlak, B., & Dustdar, S. (2026). Active Inference-Based Adaptive Routing for Heterogeneous Edge AI Services. arXiv preprint arXiv:2604.17373.
3. Chen, X., Yu, H., Ni, W., Niyato, D., Zhang, R., Wang, X., ... & Xu, S. (2026). Networking-Aware Energy Efficiency in Agentic AI Inference: A Survey. arXiv preprint arXiv:2604.07857.
4. Farooq, H., Forgeat, J., Cyras, K., Bothe, S., & Chowdhury, M. M. U. (2026, May). STAR: Smart Network-Aware AI-for-RAN Routing. In IEEE INFOCOM 2026-IEEE Conference on Computer Communications (pp. 1-6). IEEE.
5. Zheng, W., She, J., Zhang, W., Chen, Y., Chen, L., Kundu, S., ... & Yao, H. CLEAR: A Cost-Aware Routing System for Edge-Cloud Language Model Collaborative Inference.
6. Fu, Y., Qin, P., & Jiang, C. (2026). Edge AI Inference in 6G Mobile Networks: From Communication-Efficient Techniques to Task-Oriented Optimization. IEEE Communications Surveys & Tutorials.
7. Zhang, C., Dong, M., & Ota, K. (2021). Employ AI to improve AI services: Q-learning based holistic traffic control for distributed co-inference in deep learning. IEEE Transactions on Services Computing, 15(2), 627-639.
8. Parashar, P. (2026). Edge and Cloud-Based AI Inference for Big Data Processing. In Harnessing AI Inference for Intelligent Decision-Making in Real-Time Dataflows (pp. 1-28). IGI Global Scientific Publishing.
9. Jin, Y., & Yang, Z. (2025, January). Scalability optimization in cloud-based AI inference services: Strategies for real-time load balancing and automated scaling. In Proceedings of the 2025 4th International Conference on Big Data, Information and Computer Network (pp. 266-270).
10. Kodali, S. (2020). Optimizing Supply Chain Network Design with AI: Utilizing Reinforcement Learning and Advanced Analytics for Demand Forecasting, Route Optimization, and Cost Minimization. American Journal of Cognitive Computing and AI Systems, 4, 1-36.
11. Ezeagwuna, D. (2026). AI-Driven Intrusion Detection Systems for Cybersecurity. Journal of Science Technology and Social Transformation, 2(02), 37-51.
12. Mukherjee, C. (2026). AI-Based Detection of Deepfakes and Misinformation on Social Media. Euro Vantage journals of Artificial intelligence, 3(2), 9-30.
13. Mehta, S. (2026). Architecting AI Governance & ‘Agent-ready’data layer: Semantic Governance for Autonomous AI workflows. Journal of Science Technology and Social Transformation, 2(02), 29-36.
14. Ezeagwuna, D. (2026). Digital Transformation and Business Resilience: How Service Now-Powered Automation Improves Organizational Performance, Risk Management, and Customer Satisfaction. Journal of Data Analysis and Critical Management, 2(01), 48-62.
15. Rehan, H., Akram, W., & Khan, M. B. S. (2026, May). Leveraging Gated Recurrent Units for Real-Time Tracking of Student Engagement in Healthcare LMS using LLM Analytics. In 2026 Systems and Information Engineering Design Symposium (SIEDS) (pp. 1-6). IEEE.
16. Mehta, S. (2026). From Reactive cleanup to Real-time observability: Modernizing Enterprise Data Quality for RAG pipelines. Journal of Science Technology and Social Transformation, 2(01), 35-44.
17. Ezeagwuna, D. (2026). Enhancing Organizational Cyber Resilience Through the NIST Cybersecurity Framework. International Journal of Technology, Management and Humanities, 12(01), 124-140.
18. Kola, J. N. (2026, June). Secure Financial Analytics Platforms for Regulatory Compliance and Risk Management. In 2026 IEEE 18th International Conference on Computational Intelligence and Communication Networks (CICN) (pp. 794-801). IEEE.
19. Rehan, H., Zimmerman, Z., & Janjua, J. I. (2026, May). Optimizer-Driven Techniques for Retuning Prompts on LLM Backend with Migration across Models for Healthcare and Cyber Security Operations. In 2026 Systems and Information Engineering Design Symposium (SIEDS) (pp. 1-6). IEEE.
20. Kola, J. N. (2026). Enterprise Business Intelligence and Financial Analytics with a focus on Enterprise Data Architecture and Predictive Risk Integration. Adhyayan: A Journal of Management Sciences, 16(01), 20-28.
21. Ravikumar, V. (2025). Therapeutic Bot: Ethical Concerns in AI therapy for Neurodivergence. J Int Scient Re Rep.
22. Mehta, S. (2025). Enhancing Enterprise Data Governance Maturity Through Cloud Based Data Integration Frameworks in Modern Organizations. International Journal of Technology, Management and Humanities, 11(01), 59-67.
23. Rehan, H., Janjua, J. I., Ali, R. A., & Khan, T. A. (2025, December). AI-Driven DNA Insights and Public Health Data Integration for Enhancing Personal Healthcare in Critical Disease Prevention. In 2025 International Conference on AI-Driven STEM Education and Learning Technologies (AISTEMEDU) (pp. 1-6). IEEE.
24. Ezeagwuna, D. (2025). Artificial Intelligence for IT Service Management: Developing a Framework for ROI Optimization Using Service Now and Machine Learning Technologies. Well Testing Journal, 34(S4), 394-419.
25. Rehan, H., & Janjua, J. I. (2025, October). An Intelligent Data-Driven Framework for Measuring Consumer Experience in the Digital Platforms. In 2025 IEEE International Conference on Computing (ICOCO) (pp. 500-505). IEEE.
26. Mehta, S. (2025). Role of Metadata Management and Data Governance in Achieving Regulatory Compliance in Enterprise Data Ecosystems. International Journal of Technology, Management and Humanities, 11(02), 144-152.
27. Warren, Brandon. (2025). BUSINESS VALUE DRIVEN BY ENTERPRISE ARCHITECTURE TRANSFORMATION BRANDON WARREN. Journal of Tianjin University Science and Technology. 58. 10.5281/zenodo.20280991.
28. Adepoju, S. A., & Adepoju, M. A. (2024). From Portals to Case Graphs: A Reference Architecture and Benchmark for Safety Investigation Operations with Agentic Orchestration.
29. Khan, H. A. (2024). DATA-DRIVEN EPIDEMIOLOGICAL MODELING USING MACHINE LEARNING FOR DISEASE SPREAD FORECASTING AND PUBLIC HEALTH DECISION SUPPORT IN THE UNITED STATES. International Journal of Applied Mathematics, 37(6s), 178-192.
30. Kola, J. N. (2024). Longitudinal Cohort Intelligence for Self-Insured Employer Groups: A Predictive Framework for Healthcare Cost Trajectory Modeling and Proactive Risk Intervention. Journal of Information Systems Engineering & Management, 10.
31. Mehta, S. (2024). Architecting the Hub-and-Spoke Governance Model: Balancing Centralized Guardrails with Federated Domain Agility. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 16(04), 206-215.
32. Ezeagwuna, D. (2024). Enterprise Workflow Automation and Workforce Productivity: Evaluating the Economic Benefits of Service Now Adoption Across Industries. International Journal of Technology, Management and Humanities, 10(04), 299-313.
33. Kola, J. N. (2024). Longitudinal Cohort Intelligence for Self-Insured Employer Groups: A Predictive Framework for Healthcare Cost Trajectory Modeling and Proactive Risk Intervention. Journal of Information Systems Engineering & Management, 10.
34. Das, P. K., Kashem, M. A., Ferdus, Z., & Islam, S. (2019, October). Development and application of a new computerized smell generating system. In 2019 Global Conference for Advancement in Technology (GCAT) (pp. 1-5). IEEE.
35. Ferdus, M. Z., Khan, M. N. I., Islam, S., & Kashem, M. A. (2019, October). VFLT: SQA Model for Cyber Physical System. In 2019 Global Conference for Advancement in Technology (GCAT) (pp. 1-4). IEEE.
36. Thakkar, V. B. (2026). From linear logistics to neural supply chains: Predictive machine learning and the rise of autonomous supply chain intelligence. International Journal of AI, BigData, Computational and Management Studies, 7(1), 153-161.
37. Suseelan, S. (2026). LLM-Based Human-machine Teaming for Air Traffic Controllers. International Journal of Humanities and Information Technology, 8(1), 122-129.
38. Verma, A. (2022). Twin Poisoning: Analyzing the Impact of False Data Injection Attacks on Digital Twin-Based Decision Support Systems. International Journal of Technology, Management and Humanities, 8(03), 39-61.
39. Thakkar, V. B. (2026). A Unified Framework for Integrating Generative AI Across the Agile Software Development Lifecycle (SDLC). International Journal of Humanities and Information Technology, 8(2), 11-18.
40. Suseelan, S. (2026). Multimodal AI for Pilot Skill Assessment Using Physiological Signals. International Journal of Technology, Management and Humanities, 12(02), 74-83.
41. Thakkar, V. B. (2026). Asynchronous Generative AI: Automating Qualitative Data Workflows in Enterprise Systems. International Journal of Science, Technology & Society, 10(01), 1-14.
42. Ferdus, M. Z., Islam, S., & Kashem, M. A. (2019, October). An innovative load balancing cluster composition of wireless sensor networks. In 2019 global conference for advancement in technology (GCAT) (pp. 1-4). IEEE.
43. Islam, S. J., Islam, S., Ferdus, M. Z., Khan, M. N. I., Kashem, M. A., & Islam, M. S. (2020, September). Load compactness and recognizing area aware cluster head selection of wireless sensor networks. In 2020 International conference on computing and information technology (ICCIT-1441) (pp. 1-4). IEEE.
44. Thakkar, V. B. (2025). Algorithmic Trust, Governance, and Integration Bottlenecks of AI Tools in Legacy Project Environments. International Journal of Humanities and Information Technology, 7(01), 80-89.
Mazumder, P. T. (2025). Harnessing fintech innovations for anti-money laundering: a data-driven approach. Available at SSRN 5259084.
45. Suseelan, S. (2025). Big Data Analytics in Air Traffic Management: Enhancing Aviation Safety, Operational Efficiency, and Predictive Decision-Making. Journal of Science Technology and Social Transformation, 1(02), 61-73.
46. Islam, S., Khan, M. N. I., Ferdus, M. Z., Islam, S. J., & Kashem, M. A. (2020, September). Improving throughput using cooperating TDMA scheduling of wireless sensor networks. In 2020 International Conference on Computing and Information Technology (ICCIT-1441) (pp. 1-4). IEEE.
47. Naidu, K. J. (2013). Performance Optimization Of ETL Pipelines In Distributed Data Warehouse Environments: A Network-Aware Scheduling Approach. International Journal of Advance Industrial Engineering, 1(03), 63-67.
48. Ravikumar, V. (2014). Fair and optimal resource allocation in wireless sensor networks.
49. Naidu, K. J. (2014). Secure OLAP Reporting Architectures: Integrating Role-based Access Control and Query Execution Plan Optimization for Enterprise Analytical Environments. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 5(02), 155-159.
50. Kola, J. N. (2011). An Integrated Framework for Data Mining and Distributed Database Optimization in Resource-Constrained Network Environments. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 2(02), 82-86.