AMD ATOM + ATOMesh: Prefill/decode Disaggregation on ROCm

AMD has released ATOM + ATOMesh, a ROCm-native LLM serving stack that splits prefill and decode phases of inference onto separate GPUs, improving efficiency by utilizing each phase's optimal hardware. This disaggregation allows for better utilization of hardware resources, reducing bottlenecks and improving overall performance. To take advantage of this, engineers should consider implementing disaggregated inference pipelines in their ROCm-based applications. This can be achieved by separating prefill and decode phases onto different pools of GPUs, tuned for their respective bottlenecks. By doing so, they can improve the performance and efficiency of their LLM serving stacks.