The lack of tactile feedback and occlusion from visual tracking systems hinders touch interaction in Extended Reality (XR) environments. In this work, we present a method that enables touch gesture interaction on any physical surface using smartwatch-based inertial sensing. By using accelerometer data from a smartwatch, our approach captures micro-wrist movements to detect seven distinct touch gestures with 91.67% accuracy via a Long Short-Term Memory neural network. Our approach allows users to interact with XR interfaces anchored to everyday surfaces, such as tables or walls, while benefting from natural haptic feedback. We introduce a dataset collected from 20 participants to demonstrate the feasibility through a controlled study. Our fndings show that smartwatch sensing ofers a low-cost, mobile, and accurate solution for extending XR input capabilities beyond the camera’s view on physical surfaces, paving the way for more natural and privacy-preserving interaction in future XR systems.