The University of Pittsburgh’s RAMMP project integrates Meta’s open-source DINO and Segment Anything Model (SAM) to let a robotic mobility platform perceive and interact with its environment. DINO acts as a compact visual-representation backbone suited to edge devices, while SAM auto-labels training data across varied real-world conditions with minimal prompting. The team combines SAM’s outputs with DINOv2 embeddings inside a fine-tuned RF-DETR model to achieve real-time detection of objects like doors and cups, letting users direct the platform through natural language and images with reduced cognitive load.
