Voluntary actions are shaped by desired goals and internal intentions. Multiple factors, including the planning of subsequent actions and the expectation of sensory outcome, were shown to modulate kinetics and neural activity patterns associated with similar goal-directed actions. Notably, in many real-world tasks, actions can also vary across the semantic meaning they convey, although little is known about how semantic meaning modulates associated neurobehavioral measures. Here, we examined how behavioral and functional magnetic resonance imaging measures are modulated when subjects execute similar actions (button presses) for two different semantic meanings—to answer “yes” or “no” to a binary question. Our findings reveal that, when subjects answer using their right hand, the two semantic meanings are differentiated based on voxel patterns in the frontoparietal cortex and lateral-occipital complex bilaterally. When using their left hand, similar regions were found, albeit only with a more liberal threshold. Although subjects were faster to answer “yes” versus “no” when using their right hand, the neural differences cannot be explained by these kinetic differences. To the best of our knowledge, this is the first evidence showing that semantic meaning is embedded in the neural representation of actions, independent of alternative modulating factors such as kinetic and sensory features.