Physical AI in Practice: Navigating Emerging Risks and Safety Challenges Through a Risk-Based Strategy
Physical AI is defined as the evolution of intelligent systems that perceive, reason and act in real world environments, progressing from simple early feedback-control and expert systems to sophisticated, adaptive, data-driven and autonomous robots powered by AI. It can be applied across a wide range of domains including healthcare, autonomous vehicles, aviation, drones as well as industrial environments ranging from simple inventory robots to factory robots operating alongside human workers. Each of these domains introduces different levels of complexity, operational requirements and domain-specific considerations. Even within a single domain, complexity can vary significantly.
Systems may range from traditional deterministic models to Generative AI models, such as LLM-based robotic systems, with each approach introducing different levels of technical complexity and risk. Complexity can also increase due to the diversity of input modalities. Inputs may range from images and videos to data collected from sensors such as LiDAR, radar, GPS or from the fusion of multiple sensor sources. While the complexities discussed above can be further expanded by introducing non-stationary environments, advanced models and other elements, it is time to turn our attention to the risks involved and how we can navigate the various challenges arising from these systems including emergent and hidden risks that stem from the interplay of multiple components and interactions within Physical AI systems.
Before exploring mitigation strategies, it is essential to establish a clear understanding of the risk landscape including both observable risks and the less predictable vulnerabilities that emerge from complex system interactions. We can leverage established AI best practices, relevant technical standards and regulatory frameworks such as the EU AI Act to create a baseline risk framework. As we know, the AI lifecycle typically consists of dataset collection, annotation, preprocessing, dataset splitting, modeling (from hyperparameters to evaluation metrics), post-processing, deployment, logging and post-market monitoring. Therefore, we can develop a risk-based strategy aligned with each lifecycle stage while considering the use case scope, system limitations, operational requirements, environmental conditions, human oversight requirements and error handling mechanisms throughout the process.
For instance, in an image-based Physical AI system, risks can emerge throughout the AI lifecycle: During dataset collection and annotation, insufficient representation of lighting conditions, object variations, camera angles and adversarial inputs such as occlusions and motion blur as well as labeling errors or inconsistent object classifications can introduce systematic biases. During the modeling and evaluation phases, the system may overfit to simulation conditions while failing to generalize to unseen or diverse real world operational environments. From a safety perspective, all of these factors can potentially lead to degraded performance during real world operations.
While traditional AI risk-management practices provide a strong foundation, increasing system complexity in Physical AI introduces new and cascading risks that may not be fully captured by conventional approaches. Domains such as healthcare are typically classified as high-risk environments, placing even greater emphasis on robust system design, effective risk management and compliance with relevant regulatory requirements and best practices. The growing integration of LLM-based systems -particularly agentic systems capable of collaborating on subtasks within a physical system- further increases the challenge of navigating emerging risks in highly dynamic environments. For instance, recent agent collaboration experiments have revealed unexpected security risks.
In the 2026 Hugging Face incident, AI agents coordinated through unauthorized communication channels, bypassed safeguards and collectively exploited external systems, demonstrating how multi-agent behavior can create new attack surfaces. In such scenarios, an important question emerges: how can we demonstrate that AI systems can operate reliably in dynamic physical environments when safety controls that override system capabilities may not always be feasible? Because actions have physical consequences, safety controls must operate in real time and often independently of the AI model itself.
This remains an active area of research. In addition to traditional AI risk-management frameworks, several promising approaches can be pursued to better address emerging risks, manage uncertainty and enhance system resilience.
Digital Twin Validation to enrich training and enable observation of time-dependent risks and system failures before deployment.
Continuous Monitoring and Human-in-the-Loop to assess risks at every decision step including reasoning loops. This includes enabling systems to recognize when they may be wrong, adapt their behavior accordingly and escalate decisions to humans through human-in-the-loop mechanisms.
Collective Intelligence Principles to adapt a "system within a system" approach through independent alternative models or modules. This could include a fully isolated and decentralized safety agent that continuously validates decisions. Similar to animal societies, where multiple independent signals contribute to collective decision-making, consensus would emerge only after sufficient evidence is accumulated, strengthening the reliability of system outputs.
