Serving LLMs on AMD Instinct

Serving LLMs on AMD Instinct gives an agent a structured workflow for turning compatible data-center GPUs into working vLLM endpoints. It covers the operational decisions between choosing a model and confirming that its API is ready, with attention to ROCm containers, available memory and multi-GPU layouts.

Highlights

• Detects supported Instinct hardware and gathers GPU count, architecture, memory and ROCm details.
• Checks host prerequisites such as Docker access, device exposure and remote SSH connectivity.
• Estimates model-memory requirements before launch and identifies when tensor parallelism or a smaller checkpoint is needed.
• Builds vLLM container settings from model recipes and GPU-specific configuration data.
• Verifies endpoint health, performs an initial request and surfaces practical remedies for common startup failures.

The skill is aimed at platform engineers, ML infrastructure teams and developers running inference on MI300X, MI325X, MI350X or MI355X systems. It can work with a local machine or a remote host reached through SSH. Installation is available through the AMD Skills repository using the skills CLI; Bash and file-reading access are required, while no MCP server is needed.