Optimizing Neural Network Deployment Through Partitioning

Identifier:  TFM:2416

Handle: https://hdl.handle.net/20.500.11797/TFM2416

Authors:  Petkos, Theofanis

Abstract:
This work studies how to split ResNet models into k sequential groups for distributed inference. We evaluate ResNet18–152 using batch-16 ImageNet inputs in a controlled, CPU-only environment. To assess designs, we define a pipeline performance score Perf(G, n) that simulates n concurrent requests across the k-stage pipeline, respecting group availability, inter-stage dependencies, and transfer delays. Group time combines compute (sum of part runtimes) and communication (last-tensor size over bandwidth). Unless stated otherwise, weighting coefficients are fixed to α = 0.3, β = 0.4 and γ = 0.3.