practical applications for Omni robots
In recent years, a significant number of practical applications for Omni robots have been received much attention in the scientific community, especially applications in relation to navigation for autonomous robots. More importantly, based the deep learning methods are enable Omni robots to interact and navigate environments as human interaction ways. For example, by gathering data during the interaction and using end-to-end reinforcement learning [1], robots are able to learn interaction behavior through high dimensional sensory and camera information. Clearly, the visual sensor system is one of the keys to solving any tasks in relation to navigation, path detection for robots in unpredicted situations. Besides, in order tackle problem of obstacle avoidance, reinforcement learning, which plays differently from supervised and unsupervised learning branches, is becoming an efficient solution for autonomous robots nowadays [2],[3].
With the purpose of putting machines close to the human perception of artificially intelligent, the reinforcement learning approach is developed to addresses the problem of agent learning to operate in an environment through maximizing a scalar reward signal. Over the past decades, there are numerous publications that concentrate on the improvement of deep reinforcement learning (DRL). One of these is the successful game Atari 2600 or Go games in [4],[5] by observing just the screen pixels and receiving a reward calculated taking into consideration the game score. Reinforcement learning was successfully combined with a convolutional neural net to approximate the action value. Other successful works have shown in [6], Arun Nair at el displays architectures of deep reinforcement learning algorithms by optimizing approximately the Bellman equation. In terms of an autonomous car, some papers have been published in [7],[8],[9] shown that the Q learning network is trained by sampling mini-batches of experiences from buffer uniformly at random.
However, applying particularly reinforcement learning in robot setting suffers from many challenges since the high dimensionality and continuous states and actions of the robot. In a simulation, creating an accurate model robot and its environment are challenging and often require a lot of sufficient data samples. To address these challenges, the process of learning of robots is simulated and designed in Gazebo since its compatibility with the complex structure of the robot. More than that, Gazebo enables to construct of a virtual environment, which is imperative in the process of scrutinizing reinforcement learning algorithms.
This paper aims to illustrate how the Omni robot performs navigation using model-free deep Q learning to navigate in unpredicted environments. It will also explain how to obtain the policy when such a model is unknown in advance by using a virtual environment to conduct in simulation. In section II, the first architecture of the system, including a robot model that has been illustrated for the four-wheeled Omni robot. After that, the descriptions of DQN is shown in section III. Research results in 3D simulation using Gazebo is included in section IV. Finally, a conclusion is mentioned in section V