Keywords: Grasp and motion planning, Modelling and simulating humans, Physical interaction
Abstract: This paper presents a planning algorithm designed to improve cooperative robot behavior concerning human comfort during forceful human-robot physical interaction. Particularly, we are interested in planning for object grasping and positioning ensuring not only stability against the exerted human force but also empowering the robot with capabilities to address and improve human experience and comfort. Herein, comfort is addressed as both the muscular activation level required to exert the cooperative task, and the human spatial perception during the interaction, namely, the peripersonal space. By maximizing both comfort criteria, the robotic system can plan for the task (ensuring grasp stability) and for the human (improving human comfort). We believe this to be a key element to achieve intuitive and fluid human-robot interaction in real applications. Real HRI drilling and cutting experiments illustrated the efficiency of the proposed planner in improving overall comfort and HRI experience without compromising grasp stability.

Attachments: Video Attachment
Keywords: Teleoperation, Grasping and Manipulation, Physical interaction
Abstract: Existing dual-arm teleoperation systems function on one-to-one coupling of the human and robotic arms to fully exploit the user’s dexterity during bimanual tele-manipulation. While the individual coordination of the robot end-effectors can be necessary for complex and asymmetric tasks, it may result in a cumbersome user experience during symmetric bimanual tasks (e.g. manipulating and carrying objects). In this paper we propose a novel framework that includes the one-to-one direct control and a new shared autonomy strategy. The user can autonomously choose between the two, and if the new one is selected the robots move in a coordinated way, in which desired positions are extrapolated from the movements and gestures of just one users arm. These gesture commands are interpreted and handled by the control, with the purpose of unloading the users cognitive burden. Lastly, the tele-impedance paradigm, i.e., the remote control of robot impedance and position references, is applied to both controls, to improve remote physical interaction performances. The paper reports on the overall proposed architecture, its implementation and its preliminary validation trough a multi subject experimental campaign.

Attachments: Video Attachment
Keywords: Teleoperation, Humanoids for human science
Abstract: Transferring the motion from a human operator to a humanoid robot is a crucial step to enable robots to learn from and replicate human movements. The ability to retarget in real-time whole-body motions that are challenging for the humanoid balance is critical to enable human to humanoid teleoperation. In this work, we design a retargeting framework that allows the robot to replicate the motion of the human operator, acquired by a wearable motion capture suit, while maintaining the whole-body balance. We introduce some dynamic filter in the retargeting to forbid dangerous motions that can make the robot fall. We validate our approach through several experiments on the iCub robot, which has a significantly different body structure and size from the one of the human operator.

Attachments: Video Attachment
Keywords: Social interaction and acceptability, Multimodal perception
Abstract: Effectively recognising and applying emotions to interactions is a highly desirable trait for social robots. Implicitly understanding how subjects experience different kinds of actions and objects in the world is crucial for natural HRI interactions, with the possibility to perform positive actions and avoid negative actions. In this paper, we utilize the NICO robot's appearance and capabilities to give the NICO the ability to model a coherent affective association between a perceived auditory stimulus and a temporally asynchronous emotion expression. This is done by combining evaluations of emotional valence from vision and language. NICO uses this information to make decisions about when to extend conversations in order to accrue more affective information if the representation of the association is not coherent. Our primary contribution is providing a NICO robot with the ability to learn the affective associations between a perceived auditory stimulus and an emotional expression. NICO is able to do this for both individual subjects and specific stimuli, with the aid of an emotion-driven dialogue system that rectifies emotional expression incoherences. The robot is then able to use this information to determine a subjects enjoyment of perceived auditory stimuli in a real HRI scenario.

Attachments: Video Attachment
Keywords: Learning from demonstration, Modelling and simulating humans, Trajectory planning
Abstract: This paper proposes a deep learning based method for fast and responsive human-robot handover, which generates robotic motion according to observations about the human movement. Our method learns the interaction between human and a robot using a Recurrent Neural Network with Long Short-Term Memory units (LSTM-RNN) in the offline training process. The robot then uses the learned network to respond appropriately to a new human motion during the online execution. Our method is tested both on the pre-recorded data and in the real-world human-robot handover experiments. Experimental results show that our method outperforms the baseline in terms of the error of the generated robot motion. In addition, our method demonstrates a strong ability to adapt and respond to a wide variety of the human moving speeds.

Attachments: Video Attachment
Keywords: System integration, Software and hardware architecture, Industrial
Abstract: We present the collaborative humanoid robot ARMAR-6, which has been developed to perform a wide variety of complex maintenance tasks in industrial environments, collaborating with human workers. We present the hardware, software, and functional architecture of the robot as well as its current abilities. Those include the recognition of the need of help of a human worker, the execution of maintenance plans, compliant bimanual manipulation, vision-based grasping, fluent object handover, human activity recognition, natural dialog, navigation and more. We demonstrate the high level of technology readiness for real-world applications in a complex demonstration scenario, shown more than 50 times at the CEBIT 2018 exhibition.

Attachments: Video Attachment
Keywords: Novel actuation mechanisms, Novel mechanism design, Energy system for humanoids
Abstract: This paper presents the design and implementation details of an efficient robotic leg (eLeg) prototype in which series-elastic actuation is combined with adjustable parallel compliance to significantly improve its energy efficiency. The parallel actuation units are driven by secondary motors to adjust pretension of the parallel elasticity. Both monoarticulated and biarticulated actuation configurations can be employed and the leg was thus designed to permit rapid reconfiguration of its actuation units for the purpose of performing validation studies and energetic comparison of the different actuation configurations. We focus on the design procedure and implementation of the adjustable parallel actuation units, including elastic element selection, mechanism design, and force sensing capability. A design method for robots utilising the concept is presented and the experimental data are provided, that demonstrate the effectiveness of both the actuation concepts and design procedure.

Keywords: Novel actuation mechanisms, Novel mechanism design, Locomotion
Abstract: Future robots will rely more than today on high precision, better energy efficiency and safe handling (e.g. human-machine interaction). An inevitable step in the development of new robots is therefore the improvement of existing mechanisms, since better sensors and algorithms do not satisfy the demands alone. During the last three decades Parallel Redundant Mechanisms (PRM) came more in the focus of research, as they are advantageous in terms of singularity avoidance, fast movements and energy efficiency. Subsequently, yet another technology - the Variable Impedance Actuator (VIA) - emerged which proposes to change its inherent properties allowing an adaptation to its environment and to handle for example dynamic movements or shock absorptions. This work aims to create a new mechanism where a stiffness and position control for 2 degrees of freedom (DOF) is achieved by 3 actuators with flexible elements. It is thus a combination of the PRM and VIA, while taking advantage of both technologies but asking for a more sophisticated mathematical description. Practical implementation is intended for a humanoid ankle mechanism. Kinetostatic and stiffness models are derived and incorporated for the simulation of the mechanism. The simulations show that improvements in terms of singularity removal and dexterity are achieved. Furthermore, the adaptation of human like gait performances is presented.

Attachments: Video Attachment
Keywords: Novel mechanism design, Novel actuation mechanisms
Abstract: A new gripper with a Thumb Adduction/Abduction joint (TAA joint) is presented in this study. The purpose of the proposed gripper is to enable the orientation control of an object along all axes using in-hand manipulation and the last wrist joint. Using TAA joint motion, it was possible to achieve not only stable grasping but also in-hand orientation control. The gripper can produce 13 out of 16 human grasp motions according to hand taxonomy, including four of the most used hand postures. The grasping performance was evaluated using various objects. Some of them were tools for specific tasks such as electric drill, screwdriver, and hammer. The others were objects in daily life such as ball, CD, pen, cup, wallet, and card. To evaluate the capability of in-hand orientation control, the maximum ranges of rotations were also estimated using circular cross-sectional objects. The gripper can consecutively rotate the object along the roll and yaw directions, and this was demonstrated using a spherical object.

Attachments: Video Attachment
Keywords: Novel mechanism design, System integration, Humanoid kinematics
Abstract: The payload-to-weight ratio is one of the most important metrics when designing robotic wrists. A solution to maximize it and to reduce the share of the motive power required to drive the robot's links is to employ parallel kinematic mechanisms (PKMs). Indeed PKMs allow relocating distal masses closer to the robot's base actually increasing the overall payload. On the other hand, PKMs are often characterized by limited ranges of motion (ROM) and non-uniform motion in their workspace.

In this article, we considered a class of 2-DOF spherical six-bar mechanisms. We first developed the kinematic model of the system. We then tackled both the workspace limitation and uniformity issues with a numerical optimization approach. Differential evolution (a multi-objective, multivariate, gradient-free optimization method) was applied to the model of the system to explore a large space of parameter combinations.

The optimization algorithm allowed obtaining an almost uniform and large ROM (exceeding 50° on both axes). We then proceeded with the detailed design of the joint as we envision integrating it on the future releases of the iCub robot forearm-hand assemblies.

Keywords: Novel mechanism design, Novel actuation mechanisms
Abstract: In this research, we suggest a method to reduce the loss of the pump aiming to develop small and high-pressure pump for EHA. To reduce the loss of the pump, we focused on the internal leakage loss and the viscous friction loss. We formulated each loss and found that it is effective for loss reduction to make gaps inside the pump narrower and increase the discharge amount per 1 rad making the radius of the gear small. Based on the above, we developed the improved involute external gear pump. Compared with conventional trochoid pump, the internal leakage loss of developed pump is reduced by 22.4% at 10MPa and the viscous friction loss is reduced by 85.6% at 10MPa. Also, there is a problem that internal leakage increases when the internal pressure becomes high. This problem is caused by the deformation of the pump-casing under high pressure. We developed highly rigid ceramics-casing to reduce the increase of the internal leakage. Compared with the pump using aluminum-casing, the pump using ceramics-casing reduced 56.2% in internal leakage loss when the internal pressure increased from 3MPa to 6MPa.

Attachments: Video Attachment
Keywords: System integration, Novel mechanism design, Locomotion
Abstract: The performance of traditional humanoid robots is often limited by their design, with high DoF limbs and stiff actuation complicating their dynamics and impeding their ability to operate in unsupervised environments. In response to these deficiencies, this paper introduces the Non-Anthropomorphic Biped: Version 2 (NABi-V2), a bipedal robot that is a departure from the conventional humanoid paradigm in its morphology and actuation method. That is, NABi-V2 is a platform with a unique leg configuration that is designed around high torque back-drivable electric actuators that provide proprioception and force control capabilities. This paper details the concept and design of this system, and presents a simple yet robust compositional controller for performing in place, two-legged pronking, a form of continuous jumping locomotion that is typically realized with series elastic or hydraulic actuation.

Attachments: Video Attachment
Keywords: Humanoid kinematics, Trajectory planning
Abstract: Robot performance measures are essential tools for quantifying the ability to execute manipulation tasks. Typically, these measures focus on the system's geometric structure and how it impacts the transformation from joint to Cartesian space. In this paper, we propose a new method to evaluate the robot's performance that considers both the system's geometric structure and the presence of obstacles close to or in contact with the robot. This method reduces the manipulator's joint velocity limits by deforming the manipulability polytope to account for obstacles. These constraints are then propagated throughout the chain to get a more representative measure of the end effector's velocity capabilities. The proposed method leads to improved understanding of the robot's capacities in a constrained environment.

Attachments: Video Attachment
Keywords: Body balancing, Concept and strategy learning, Trajectory planning
Abstract: Two key questions in the context of stepping for push recovery are where to step and how to step there. In this paper we present a fast and computationally light-weight approach for capture stepping of full-sized humanoid robots. To this end, we developed an efficient parametric step motion generator based on dynamic movement primitives (DMPs) learnt from human demonstrations. Simulation-based reinforcement learning (RL) is used to find a mapping from estimated push parameters (push direction and intensity) to step parameters (step location and step execution time) that are fed to the motion generator. Successful omni-directional capture stepping for 89% of the test cases with pushes from various directions and intensities is achieved with minimal computational effort after 500 training iterations. We evaluate our method in a dynamic simulation of the ARMAR-4 humanoid robot.

Keywords: Locomotion, Locomotion planning
Abstract: Generating and executing whole-body motions for humanoid robots remains a challenging research question. In this paper we present an approach which combines human motion data and QP-based control to generate humanoid motion. Following the contacts-before-motion paradigm, we first generate a sequence of stances based on our previous work on data-driven generation of whole-body multi-contact pose sequences from human motion data and their mapping to the target robot kinematics. In this paper, we address the next step of closed-loop execution of stance sequences based on QP controllers. We evaluated the approach in simulation on the humanoid robot ARMAR-4 and HRP4. The results show that our approach can be used to successfully execute stance sequences generated by our previous work and thus the viability of learning locomotion patters from human demonstrations.

Keywords: Humanoid dynamics, Body balancing
Abstract: Inspired by biological systems, robots that exploit the natural dynamics of compliant joints are developed in recent years to obtain stable and efficient locomotion. In these robots,series elastic actuator(SEA) is widely used due to its compliant property and energy storage capacity. However, robots that are equipped with SEA have drawbacks of substantial delay and limited bandwidth. Additionally, high speed locomotion also engenders severe vibration and cause noise pollution in posture measurement of the robot. These inevitable features make the efficient robots hard to demonstrate precise control and perform dynamic balance. To cope with these problems, beside traditional hopping and foot hold selection algorithms, two methods are proposed in this paper for consecutive hopping:(1)an position controller which generates active damping to stabilize the joint position;(2)an learning algorithm for body balance control. The learning algorithm discretizes the continuous control problem into phases and adopts integration form of body dynamics to maintain balance. Instead of empirically tuning the control parameters, model identification and learning algorithms are employed to automatically tune these proposed controllers. Experiments were conducted on SEA based single leg robot by swinging leg between two demanded position and maintaining body balance during consecutive hopping. By combining the proposed algorithms, stable and efficient hopping was implemented.

Attachments: Video Attachment
Keywords: Multimodal perception, Locomotion
Abstract: Several humanoid robots will require to navigate in unsafe and unstructured environments, such as those after a disaster, for human assistance and support. To achieve this, humanoids require to construct in real-time, accurate maps of the environment and localize in it by estimating their base/pelvis state without any drift, using computationally efficient mapping and state estimation algorithms. While a multitude of Simultaneous Localization and Mapping (SLAM) algorithms exist, their localization relies on the existence of repeatable landmarks, which might not always be available in unstructured environments. Several studies also use stop-and-map procedures to map the environment before traversal, but this is not ideal for scenarios where the robot needs to be continuously moving to keep for instance the task completion time short. In this paper, we present a novel combination of the state-of-the-art odometry and mapping based on LiDAR data and state estimation based on the kinematics-inertial data of the humanoid. We present experimental evaluation of the introduced state estimation on the full-size humanoid robot WALK-MAN while performing locomotion tasks. Through this combination, we prove that it is possible to obtain low-error, high frequency estimates of the state of the robot, while moving and mapping the environment on the go.

Attachments: Video Attachment
Keywords: Novel mechanism design, Software and hardware architecture
Abstract: The design of humanoid robots is a complex, challenging and time-consuming task. Due to conflicting requirements, such as human-like capabilities within human dimensions, the design of humanoid robots relies highly on the experience and expert knowledge of the engineers. This paper presents an expert system framework that allows to store this knowledge in order to reuse it for the systematic design of humanoid robot components. Based on user requirements, the system executes a multi-stage reasoning on an ontological knowledge base: Partial solutions are generated by integrating existing catalog components into potential concept solutions. After checking logical and physical constraints as well as calculating properties, these partial solutions are either discarded or combined in a bottom-up way to generate valid solutions that are then visualized by a user interface. We evaluate the developed system in terms of its capability to reproduce available solutions for state-of-the-art sensor-actuator units used in several robots as well as its capability to optimize the design of such units.

Attachments: Video Attachment
Keywords: Grasping and Manipulation, Novel mechanism design, Physical interaction
Abstract: We present a novel class of tendon-actuated soft robots, which promise to be low-cost and accessible to non-experts. The primary structure of the robot consists of flexible foam, and so we term the robots created using our approach ``foam robots.'' A foam robot moves by driving servo mounted winches that contract (or slacken) tendons routed through the robots textile skin. We provide a methodology for fabricating these types of robots and go on to fabricate several `foam robots' in the form of multi-fingered hands and perform various experiments and demonstrations to illustrate the robust applications of these robots to tasks such as dexterous manipulation.

Keywords: Humanoid dynamics, Physical interaction, Body balancing
Abstract: Impedance control of humanoid robots, a form of compliant control, allows them to move in a fashion similar to humans and increase the safety of interactions with humans or the environment. In low stiffness impedance control, gravitational forces will cause the robot to deviate significantly from the desired position. Thus, a gravity compensation term in the joint motor torque command is required to counteract gravitational forces. Ground reaction forces are sometimes used to estimate the gravity compensation torque required for each joint. In this paper, a novel method to estimate contact forces by using model mass properties and relative force and torque sensor data of each contact point with respect to all loaded limbs is proposed. This simple and straightforward method, called the proportional method, is able to resolve internal forces arising from closed-loop kinematic chains in multi-contact situations, for example the double support phase of bipedal robots, without optimization. The proposed method is also more robust to sensor error and is able to implicitly distinguish between gravitational and external forces for impedance control. Simulations and experiments using the humanoid robot HRP-4 are performed to validate the proposed method.

Attachments: Video Attachment
Keywords: Locomotion, Body balancing, Humanoid dynamics
Abstract: This paper presents an experimental evaluation of passivity-based whole-body motion control framework for compliant walking. The controller computes joint torques without requiring much computation cost and contact force measuring. Instead of limiting the walking speed slow (static walking), in this work we specifically address the difficulties of walking on unstable and uneven ground. No terrain information is used in the experiments, that is, the ground is assumed to be flat, and the desired motion trajectories are given offline. With this setup we evaluate the terrain adaptability by force control alone. The controller is applied to our torque-controllable hydraulic humanoid robot, TaeMu. The robot could walk on a rocker board stably, and even climbed the small step with a little modification of the controller (quasi-dynamic walking).

Keywords: Locomotion planning, Locomotion, Task planning
Abstract: This paper studies a model-based feedback controller which can generate limit cycle walking at target walking speed, and identify the physical parameters through neural network. First, a combined rimless wheel is developed, and the feedback control is proposed by dynamic planning its equation of motion. Second, the numerical simulations are conducted to analyse walking speed and other properties when the physical parameters are assumed unknown and the prediction parameters are used instead. The controller has a certain adaptability to the prediction error, and the target walking speed can be generated with little error (0.001%). Finally, based on the model-based properties of the control, the physical parameters can be predicted through a proposed neural network model with an average error of 2%. In general, the model-based feedback controller provides us a new approach for simultaneously controlling walking speed and identifying physical parameters.

Keywords: Modelling and simulating humans, Medical, health and mental care
Abstract: In this study, we propose a muscle fatigue and recovery model with an energy supply system and physiological basis. Fatigue level is evaluated by maximum muscle contraction force. In the energy supply system, the amounts of aerobic and anaerobic respirations are calculated based on oxygen consumption rate. The variation of related chemical compounds, like lactate and glucose, can be also obtained, which are used to predict the fatigue level. The proposed model is verified by an application to human arm movements. Comparison between the estimated and the measured maximum muscle forces demonstrates the effectiveness of the model.

Attachments: Video Attachment
Keywords: Modelling and simulating humans, Learning from demonstration, Medical, health and mental care
Abstract: The ability to predict the future intended movement is crucial for collaborative robots to anticipate the human actions and for assistive technologies to alert if a particular movement is non-ergonomic and potentially dangerous for the human health. In this paper, we address the problem of predicting the future human whole-body movements given early observations.

We propose to predict the continuation of the high-dimensional trajectories mapped into a reduced latent space, using autoencoders (AE). The prediction is based on a probabilistic description of the movement primitives (ProMPs) in the latent space, which notably reduces the computational time for the prediction to occur, and hence enables to use the method in real-time applications. We evaluate our method, named AE-ProMPs, for predicting future movements belonging to a dataset of 7 different actions performed by a human, recorded by a wearable motion tracking suit.

Keywords: Novel actuation mechanisms, Novel mechanism design, System integration
Abstract: Biped robots require a high power to be provided alternately on their two legs while walking, hopping, and running. However, the mounting of high-power and large electrical motors is challenging in conventional mechanical transmission systems because of space limitations. To address this issue, we employ herein a combination of hydraulic and transmission systems with an independent driving mode and a power-shared driving mode. In the independent driving mode, an actuator can be independently controlled based on flow-control, and pressure loss can be reduced. In the power-shared driving mode, actuators can also be controlled based on flow-control, and this mode allows the motor power of the left and right legs to be shared. We also employ a simulation to evaluate the proposed novel system and confirm that the motor power could be reduced by 35.6% for the hopping movement. This result shows that the rated output of the required motor can be reduced, and the selection of smaller and lighter motors is possible for installation in biped robots.

Attachments: Video Attachment
Keywords: Novel actuation mechanisms, Locomotion, Physical interaction
Abstract: The backdrivability of joints is a critical requirement for the robots that perform tasks in uncertain environments. While series elastic actuators are intrinsically backdrivable, their control bandwidth is limited by the low resonant frequency of the elastic component. To simultaneously realize both of the backdrivability and high control bandwidth, Electro-Hydrostatic Actuator (EHA) is a solution. Based on this idea, we developed the fully electro-hydrostatically driven humanoid robot Hydra, while its evaluation was limited to the joint level one. In this paper, we present evaluations of its whole-body control performance, including the locomotion. This is the first time to report a bipedal locomotion by an EHA driven humanoid. We first confirm that Hydra can realize a position feedback control with enough stiffness to realize a position control based locomotion. Secondly, we show that the joint backdrivability can suppress the effect of a disturbance applied to the distal part of the robot on the whole-body motion. As the result, we realized a torque control based locomotion with both a proper COM stabilization and nullspace compliance.

Keywords: Locomotion, Body balancing
Abstract: This paper proposes a novel biped stepping control which does not depend on the time-defined trajectory. The up-down motion of the foot is determined by referring to the LIPM potential, which is defined in the paper, based on the phase-space analysis of the center of mass (COM). The motion rate of the COM relative to the Zero-Moment Point (ZMP) is represented as the gradient of the potential, and the potential monotonously decreases from positive to negative during one step. The dominant component of the COM movement turns from convergent mode to divergent mode at the zero potential, and thus, the positive potential encourages the lifting-up and the negative potential alerts the necessity of touch-down. This emerges a stable alternate stepping of the feet by combining with a self-exciting oscillation of the COM and the ZMP, which was also proposed by one of the authors. The controller provides robots with flexibility against disturbances since it does not rely on any pre-defined referential motion trajectory. Computer simulations show that this idea is valid for a bipedal foot controller.

Attachments: Video Attachment
Keywords: Learning from demonstration, Physical interaction
Abstract: Intelligent robotic assistants can potentially improve the quality of life for elderly people and help them maintain their independence. However, the number of different and personalized tasks render pre-programming of such assistive robots prohibitively difficult. Instead, to cope with a continuous and open-ended stream of cooperative tasks, new collaborative skills need to be continuously learned and updated from demonstrations. To this end, we introduce an online learning method for a skill library of collaborative tasks that employs an incremental mixture model of probabilistic interaction primitives. This model chooses a corresponding robot response to a human movement where the human intention is extracted from previously demonstrated movements. Unlike existing batch methods of movement primitives for human-robot interaction, our approach builds a library of skills online, in an open-ended fashion and updates existing skills using new demonstrations. The resulting approach was evaluated both on a simple benchmark task and in an assistive human-robot collaboration scenario with a 7DoF robot arm.

Attachments: Video Attachment
Keywords: Grasping and Manipulation
Abstract: Bimanual manipulation is fundamental for humanoid robots. It has gained a lot of attention in robotics research as a key ability towards versatile behavior. To achieve such behaviors in real-world tasks, bimanual controllers must be stable and simple to implement. On the other hand, admittance and impedance control frameworks are well-known for their efficiency in robot's manipulation tasks which require compliant motions eg for physical human-robot interactions. Based on these frameworks, we propose a new control framework, the Projected Force-Admittance Control (PFAC), for compliant bimanual manipulation tasks. By analyzing the load distribution in bimanual tasks using grasp mapping technique, the controller uses the projected constraint force, which, together with the actuation force given by the PI controller, are fed into an admittance control framework, and finally provides the virtual target pose to an impedance controller that can be modeled as a mass-spring-damper system. With this control strategy, we ensure motion synchronization and target force regulation under external perturbations and/or while tracking a trajectory. We demonstrate the stability and usability of the controller in several experiments with the humanoids robot ARMAR-6. Combining it with movement primitives approaches such as Dynamic Movement Primitive (DMP), a variety of compliant bimanual tasks are implemented and evaluated.

Attachments: Video Attachment
Keywords: Locomotion, Grasping and Manipulation, Body balancing
Abstract: This paper studies the method of target recognition and heavy load trolley control in humanoid robot pushing operation. In this paper, a piecewise fitting monocular vision ranging method is proposed to achieve the target search and location of humanoid robot NAO. Based on vision localization, the research of humanoid robot pushing operation is carried out, and the posture closed loop control of heavy load robot is carried out. The experimental results show the effectiveness and accuracy of the method of single eye distance measurement, target location and recognition and the control of the motion of the cart, and successfully completed the light load operation and heavy load operation of the humanoid robot dual-arm trolley.

Attachments: Video Attachment
Keywords: Concept and strategy learning, Body balancing
Abstract: Generating complex whole-body movements for humanoid robots is now most often achieved with multi-task whole-body controllers based on quadratic programming. To perform on the real robot, such controllers often require a human expert to tune or optimize the many parameters of the controller related to the tasks and to the specific robot, which is generally reported as a tedious and time consuming procedure. This problem can be tackled by automatically optimizing some parameters such as task priorities or task trajectories, while ensuring constraints satisfaction, through simulation. However, this does not guarantee that parameters optimized in simulation will also be optimal for the real robot. As a solution, the present paper focuses on optimizing task priorities in a robust way, by looking for solutions which achieve desired tasks under a variety of conditions and perturbations. This approach, which can be referred to as domain randomization, can greatly facilitate the transfer of optimized solutions from simulation to a real robot. The proposed method is demonstrated using a simulation of the humanoid robot iCub for a whole-body stepping task.

Attachments: Video Attachment
Keywords: Teleoperation, Physical interaction
Abstract: The improved adaptability of a robotic teleoperation system to unexpected disturbances in remote environments can be achieved by compliance control. Nevertheless, complying with all types of interaction forces while performing realistic manipulation tasks may deteriorate the teleoperation performance. For instance, the loading effect of the objects and tools that are held and manipulated by the robot can introduce undesired deviations from the reference trajectories in case of low-stiffness (or high payload) teleoperation. Although this can be addressed by updating the robot dynamics with the external loading effect, a sudden loss of the object may also generate undesired and potentially dangerous robot behaviours. To address this problem, we propose a novel and self-adaptive teleoperation framework. The method uses the feedback from robot's force sensors to recognize the interaction aspects that must be compensated by robot dynamics. Thanks to this on-line compensation, the slave robot reduces the tracking error with respect to the commanded motion by the human operator, while performing complex interactive tasks without the haptic feedback. The robot local controller also includes an energy tank based passivity paradigm to be able to manage unexpected collisions or a contact loss without resulting in an unsafe behaviour. We validate the proposed approach by experiments on a torque-controlled robotic arm performing manipulation tasks that require both object manipulation and environment interaction.

Keywords: Body balancing, Locomotion
Abstract: In this paper, the intuitive idea of tilt is formalised into the rigorous concept of tilt rotations. This is motivated by the high relevance that pure tilt rotations have in the analysis of balancing bodies in 3D, and their applicability to the analysis of certain types of contacts. The notion of a 'tilt rotation' is first precisely defined, before multiple parameterisations thereof are presented for mathematical analysis. It is demonstrated how such rotations can be represented in the so-called tilt phase space, which as a vector space allows for a meaningful definition of commutative addition. The properties of both tilt rotations and the tilt phase space are also extensively explored, including in the areas of spherical linear interpolation, rotational velocities, rotation composition and rotation decomposition.

Attachments: Video Attachment
Keywords: Grasp and motion planning, Humanoid kinematics, Body balancing
Abstract: This paper presents a method for motion synthesis using Functional Principal Component Analysis (Functional PCA) to generate complex humanoid robot motions in a low-dimensional space while considering physical consistency. Since each motion can be expressed by a point in a space called FPC space, this method allows blending different motions. For more complex motion synthesis, we introduce a novel framework to synthesize blended motions by configuring a local FPC space and a global FPC space. This method enable to merge data with considering data features. However, physical consistency was not ensured in our previous work, we apply optimization under constraints after synthesis. We show the dynamic feasibility and the feature of synthesized blended motions and also an interesting observation opening to the possibility to generate variety motions from a few motion data in a local space without taking cost and time.

Keywords: Industrial, Grasp and motion planning, Novel mechanism design
Abstract: This paper presents an approach for systematic evaluation of robotic end effectors for an industrial use case that handles delicate, deformable, non-regular objects such as fruits and vegetables. To handle these objects, soft under-actuated hands are the most promising technology so far. However, the approach directions suitable for grasping objects are not usually easy to establish due to the under-actuation effect; therefore, we propose an experimental protocol to serve as framework for data collection, which aims to assess the best directions for grasping using a particular end effector. This protocol, focused on reproducibility and comparability, allows for a better understanding of how a particular hand embodiment influences grasping success for individual products. The protocol can also be used as an effective tool for redesign, and it was applied to evaluate two well-known soft hands (RBO Hand and Pisa/IIT SoftHand) for handling groceries.

Attachments: Video Attachment
Keywords: Humanoid dynamics, Learning from demonstration, Body balancing
Abstract: In task-based inverse dynamics control, reference accelerations used to follow a desired plan can be broken down into feedforward and feedback trajectories. The feedback term accounts for tracking errors that are caused from inaccurate dynamic models or external disturbances. On underactuated, free-floating robots, such as humanoids, good tracking accuracy often necessitates high feedback gains, which leads to undesirable stiff behaviors. The magnitude of these gains is anyways often strongly limited by the control bandwidth. In this paper, we show how to reduce the required contribution of the feedback controller by incorporating learned task-space reference accelerations. Thus, we i) improve the execution of the given specific task, and ii) offer the means to reduce feedback gains, providing for greater compliance of the system. In contrast to learning task-specific joint-torques, which might produce a similar effect but can lead to poor generalization, our approach directly learns the task-space dynamics of the center of mass of a humanoid robot. Simulated and real-world results on the lower part of the Sarcos Hermes humanoid robot demonstrate the applicability of the approach.

Attachments: Video Attachment
Keywords: Body balancing, Locomotion
Abstract: In this paper we report the implementation and the experimental validation of a controller to provide reactive walking gait capabilities of bipedal robots during the execution of predefined walking patterns. The proposed method is a cascade controller design to cope with external disturbances and to increase the robot stability. IMU states are used as inputs to generate modifications of the feet and the Center of Mass trajectories of the predefined walking gait. The method increases the walking stability minimizing the errors due to small terrain variations and external disturbances. The effectiveness of the proposed controller is validated in simulation and in real implementation on the full-body humanoid robot COMAN+.

Attachments: Video Attachment
Keywords: Cognitive development, Social interaction and acceptability, Multimodal perception
Abstract: Personality is nothing but individual differences in the way we tend to think, feel and behave. It is ingrained in our basic instincts which tend to answer the question of why people differ in behavioral aspects in our day-to-day life. The assessment of personality traits is highly significant in human-human interaction. However, the topic has not been studied extensively in the context of human-robot interaction. This study focuses on the significance of nonverbal cues with respect to personality traits. A supervised learning approach has been used to recognize 3 personality traits of the big five model namely extroversion, agreeableness and neuroticism traits. Nonverbal cues such as head gestures, postures, proxemics, facial expressions and bodily cues are used to construct a feature vector for classification. A humanoid robot, ROBIN, is used for the assessment of personality traits in different scenarios. Sequences are labeled with the help of a psychology expert. The system shows above 90% accuracy in the automatic assessment of personality traits.

Keywords: Prostheses & Ortheses, Exoskeletons and assistive devices, Physical interaction
Abstract: The correct estimation of human hand kinematics has received a lot of attention in many research fields of neuroscience and robotics. Not surprisingly, many works have addressed hand pose reconstruction (HPR) problem and several technological solutions have been proposed. Among them, Inertial and Magnetic Measurement Unit (IMMU) based systems offer some elegant characteristics (including cost-effectiveness) that make these especially suited for wearable and ambulatory HPR. However, what still lacks is an exhaustive characterization of IMMU-based orientation tracking algorithms performance for hand tracking. In this work, we have developed an experimental protocol to compare the performance of three of the most widely adopted HPR computational techniques, i.e. extended Kalman filter (EKF), Gauss-Newton with Complementary filter (CF) and Madgwick filter (MF), on the same dataset acquired through a IMMU-based sensing glove. Quality of the algorithms has been benchmarked against the ground truth measurement provided by an optical motion tracking system. Results suggest that performance of the three algorithms are similar, yet the MF algorithm appears to be slightly more accurate in reconstructing the individual joint angles during static trials and to be the fastest one to run.

Keywords: Physical interaction, Industrial, Task planning
Abstract: Industry 4.0 envisions the integration of flexible and quickly reconfigurable robotic systems in assembly lines. This has led to the development of light-weight and adaptive collaborative robots with limited power and payload constraints. Hence, repetitive tasks or those that demand high forces may exceed such limits, resulting in robot damage and lost productivity. To address this problem, we propose a novel framework to prolong the lifetime of collaborative robots while guaranteeing the desired level of productivity. To address this, we propose a method that minimises a function of robot fatigue, an index able to model the robot motor usage at the joint level. This index features the desired external force, task duration, hardware parameters, and fatigue history. Moreover, future tasks are considered through fatigue thresholds imposed on specific joints, computed according to the robot safety requirements. We proved the effectiveness of the approach by comparing its results in terms of fatigue and torque with the well-known minimum effort approach. The results showed that our method ensures that the fatigue thresholds are not exceeded.

Attachments: Video Attachment
Keywords: Deep Learning, Task planning, Trajectory planning
Abstract: Performing dual arm coordinated (reachability)tasks in humanoid robots require complex planning strategies and this complexity increases further, in case of humanoids with articulated torso. These complex strategies may not be suitable for online motion planning. This paper proposes a faster way to accomplish dual arm coordinated tasks using methodology based on Reinforcement Learning. The contribution of this paper is twofold. Firstly, we propose DiGrad (Differential Gradients), a new RL framework for multi-task learning in manipulators. Secondly, we show how this framework can be adopted to learn dual arm coordination in a 27 degrees of freedom (DOF) humanoid robot with articulated spine. The proposed framework and methodology are evaluated in various environments and simulation results are presented. A comparative study of DiGrad with its parent algorithm in different settings is also presented.

Keywords: Visual perception, Deep Learning, Grasping and Manipulation
Abstract: Many objects in a human-made environment have elongated shapes for easy manipulation and grasping. As humanoid robots are working in this environment, they require proper sensing and perception of such objects. Current approaches are providing mainly the perception of rigid objects, but many everyday items are non-rigid and more challenging to track due to their substantial shape variability. We want the robots to be able to grasp and manipulate thin, elongated, deformable objects. We propose a system based on the Deep Neural Network that can predict the bend angle of such objects using the single RGB image only. In our paper, we present the proposed neural network architecture used for prediction of the bending angle and finding the elongated shape in images with a cluttered background together with the dataset used for training. We observed that the proposed system even though it was trained on synthetic data was able to perform well on real data. The proposed architecture also provide us with the ability to hallucinate how the deformable pipe with any initial bend would look like when subjected to the arbitrary bend angle. Our findings have more profound consequences than the above mentioned. We were able to show that the proposed Encoder-Decoder neural network architecture has the textit{interpretable latent vector element} for describing a textit{measurable physical bend angle}. Moreover, we allow bending arrows to be situated out of the image plane. In the future work, we are planning to extend the current approach with the prediction of the full 3d shape of the elongated object from a single RGB image.

Attachments: Video Attachment
Keywords: Grasping and Manipulation, Tactile perception, Body balancing
Abstract: Humanoid robots are designed to perform tasks in the same way than humans do. One of these tasks is to act as a waiter serving drinks, food, etc. Transporting all this stuff can be considered as a manipulation task. In this application the objects are transported over a tray, without grasping them. The consequence is that the objects are no firmly attached to the robot, which is the case in grasping. Then, complexity of robotics grasping is avoided but stability issues arise. The problem of keeping balance of object transported by a robot over a tray is presented in tis paper. The approach presented is founded in the computation of the Zero Moment Point (ZMP) of the object, which is modeled as a three dimensional Linear Inverted Pendulum (3D-LIPM). The use of force-torque sensors located at the wrist enables ZMP computation. But the main problem to be solved is how the robot should react when the object loss balance. One strategy is to rotate the tray to counteract the rotation of the object. This rotation has to be proportional to the ZMP variation and the object's rotation angle. This issue is solved applying the concept of three dimensional dynamic slopes. It helps to avoid kinematic problems and make balance computation independent from the angle of the tray.

Attachments: Video Attachment
Keywords: Physical interaction, Social interaction and acceptability, Grasping and Manipulation
Abstract: For robots to share the environment and cooperate with humans without barriers, we need to guarantee safety to the operator and, simultaneously, to maximize the robot's usability. Safety is typically guaranteed by controlling the robot movements while possibly in physical contact with the operator, objects or tools. If possible, also the safety of the robot must be guaranteed. Not less importantly, as the complexity of the robots and their skills increase, usability becomes a concern. Social interaction technologies can save the day by enabling natural human-robot collaboration.

In this paper we show a possible integration of physical and social Human-Robot Interaction methods (pHRI and sHRI respectively). Our reference task is {em object hand-over}. We test both the case of the robot initiating the action and, vice versa, the robot receiving an object from the operator. Finally, we discuss possible extension with higher-level planning systems for added flexibility and reasoning skills.

Attachments: Video Attachment
Keywords: Body balancing
Abstract: This paper proposes a capturability analysis method for fall avoidance of bipedal robots under arbitrary disturbances. Based on a dynamical model of the planar movement of the center-of-mass, capture region is computed numerically by discretizing the state space and the set of control inputs. The proposed method is able to handle a number of practically important elements of fall avoidance such as the relation between stride length and step duration, and kinematic limitation of foot placement, which have been neglected in conventional studies for simplification. The developed fall-avoidance controller utilizes precomputed capturable regions to filter reference foot placements produced by a foot-step planner to ensure fall-avoidance with small online computation time. Capture regions computed by the proposed method are compared with the conventional ones in case studies. The performance of the proposed fall-avoidance controller is evaluated in simulations.

Attachments: Video Attachment
Keywords: Locomotion planning, System integration, Exoskeletons and assistive devices
Abstract: In this paper, we propose a novel method for retargeting human movements including the steps taken by a humanoid robot. For applications, such as wearable device evaluation by a humanoid, it is necessary to reproduce the original human motions as closely as possible and to generate motions that can be realized on a real robot without the robot losing its balance. The proposed method features the optimization framework that integrates the walking pattern generation based on the linear inverted pendulum model whose parameters are optimized to ensure both human likeness and feasibility of the generated motions. The effectiveness of the retargeting method is validated by experiments that reproducing the measured human working motion on the humanoid HRP-4.

Attachments: Video Attachment
Keywords: Visual perception, Tele-experience and tele-presence using humanoids, Home, field, space, underwater
Abstract: Falling down is one of the biggest problems for humanoids robots, while dynamic obstacles which occlude environment features, lead to a hard problem for visual Simultaneous Localization and Mapping (SLAM) approaches. These two problems block the SLAM method applications for humanoids. Humans are often considered as moving obstacles or moving targets in the humanoids working spaces and human-robot interaction cases. Thus, how to detect and process moving humans is a crucial issue of humanoids SLAM. In this paper, we propose a robust dense RGB-D environment reconstruction method for humanoids working in dynamic humans space. The proposed approach efficiently detects humans and fast reconstructs the static environments through deep learning-based human body detection, and then implement a graph-based segmentation the RGB-D point clouds, which separates detected moving human from the static environments. Finally, the separated static environments are aligned with using state-of-the-art frame-to-model scheme. Experimental results on both public benchmark and a newly developed HRP-4 humanoids SLAM dataset indicated that the proposed approach achieves outstanding performance in full dynamic environments.

Attachments: Video Attachment
Keywords: Grasp and motion planning, Trajectory planning, Deep Learning
Abstract: Manipulation in clutter requires solving complex sequential decision making problems in an environment rich with physical interactions. The transfer of motion planning solutions from simulation to the real world, in open-loop, suffers from the inherent uncertainty in modelling real world physics. We propose interleaving planning and execution in real-time, in a closed-loop setting, using a Receding Horizon Planner (RHP) for pushing manipulation in clutter. In this context, we address the problem of finding a suitable value function based heuristic for efficient planning, and for estimating the cost-to-go from the horizon to the goal. We estimate such a value function first by using plans generated by an existing sampling-based planner. Then, we further optimize the value function through reinforcement learning. We evaluate our approach and compare it to state-of-the-art planning techniques for manipulation in clutter. We conduct experiments in simulation with artificially injected uncertainty on the physics parameters, as well as in real world tasks of manipulation in clutter. We show that this approach enables the robot to react to the uncertain dynamics of the real world effectively.

Attachments: Video Attachment
Keywords: System integration, Software and hardware architecture, Deep Learning
Abstract: Humanoid robotics research depends on capable robot platforms, but recently developed advanced platforms are often not available to other research groups, expensive, dangerous to operate, or closed-source. The lack of available platforms forces researchers to work with smaller robots, which have less strict dynamic constraints or with simulations, which lack many real-world effects. We developed NimbRo-OP2X to address this need. At a height of 135cm our robot is large enough to interact in a human environment. Its low weight of only 19kg makes the operation of the robot safe and easy, as no special operational equipment is necessary. Our robot is equipped with a fast onboard computer and a GPU to accelerate parallel computations. We extend our already open-source software by a deep-learning based vision system and gait parameter optimisation. The NimbRo-OP2X was evaluated during RoboCup 2018 in Montreal, Canada, where it won all possible awards in the Humanoid AdultSize class.

Attachments: Video Attachment
Keywords: System integration, Home, field, space, underwater, Humanoid dynamics
Abstract: In order to make humanoid robots work in the real world, it is necessary to construct a robot system that can be operated without any crane support from start to finish. This paper deals with crane-less servo-off operation of life-sized humanoid robot in which a robot safely turns off / on the joint servo without relying on external physical support. We organize the necessity and difficulty of life-sized humanoid servo-off and introduce a post-evaluation based heuristic procedure of generating servo-off posture. By generated servo-off posture and scripted transition motion, we demonstrate the crane-less servo-off operation with real life-sized humanoid robots in several scenarios.

Attachments: Video Attachment
Keywords: Trajectory planning, Body balancing
Abstract: It is in process to build robust robotics system enabling whole-body multi-contact motion. In this paper, we have experiments on knee-contact motions to preliminary investigate motion planning algorithm to generate whole-body multi-contact behavior. Our motion interpolator is goal-oriented in that the interpolator does not specify detailed contact constraints such as fixed contact point on link, friction cone constraints and timing of contact switching. The goal-oriented feature enables to generate complex contact transition including sliding, rotating and dynamic contact transition. The interpolator generates whole-body trajectory to achieve goal state considering physical feasibility such as whole-body dynamics, collision, and joint torque limitations by using dynamics simulator. Further, the generated knee-contact motions are achieved by actual humanoid robot RHP4B to check difference between simulated motion and actual result.

Attachments: Video Attachment
Keywords: Grasping and Manipulation, Tactile perception, Deep Learning
Abstract: Changing the grasping posture of objects within a robot hand is hard to achieve, especially if the objects are of various shape and size. In this paper we use a neural network to learn such manipulation with variously sized and shaped objects. The TWENDY-ONE hand possesses various properties that are effective for in-hand manipulation: a high number of actuated joints, passive degrees of freedom and soft skin, six-axis force/torque (F/T) sensors in each fingertip and distributed tactile sensors in the soft skin. The object size information is extracted from the initial grasping posture. The training data includes touch states and the object information. After training the neural network with the data, the robot is able to manipulate objects of not only trained but also untrained sizes and shape. The results show the importance of size and touch information. Importantly, the features extracted by a stacked autoencoder could reduce the number of required training samples for in-hand manipulation.

Attachments: Video Attachment
Keywords: Body balancing, Deep Learning, Locomotion
Abstract: This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors, i.e., ankle, hip, foot tilting, and stepping strategies. The policy is trained in a physics simulator with realistic setting of robot model and low-level impedance control that are easy to transfer the learned skills to real robots. The advantage over traditional methods is the integration of high-level planner and feedback control all in one single coherent policy network, which is generic for learning versatile balancing and recovery motions against unknown perturbations at arbitrary locations (e.g., legs, torso). Furthermore, the proposed framework allows the policy to be learned quickly by many state-of-the-art learning algorithms. By comparing our learned results to studies of preprogrammed, special-purpose controllers in the literature, self-learned skills are comparable in terms of disturbance rejection but with additional advantages of producing a wide range of adaptive, versatile and robust behaviors.

Attachments: Video Attachment
Keywords: Physical interaction, Novel mechanism design, Humanoids for human science
Abstract: By the recent spread of machine learning in the robotics field, a humanoid that can act, perceive, and learn in the real world through contact with the environment needs to be developed. In this study, as one of the choices, we propose a novel humanoid TWIMP, which combines a human mimetic musculoskeletal upper limb with a two-wheel inverted pendulum. By combining the benefit of a musculoskeletal humanoid, which can achieve soft contact with the external environment, and the benefit of a two-wheel inverted pendulum with a small footprint and high mobility, we can easily investigate learning control systems in environments with contact and sudden impact. We reveal our whole concept and system details of TWIMP, and execute several preliminary experiments to show its potential ability.

Attachments: Video Attachment
Keywords: Concept and strategy learning, Learning from demonstration, Deep Learning
Abstract: In order to operate in everyday human environment, humanoids robots will need to autonomously learn and adapt their actions, using among other reinforcement learning methods (RL). A common challenge in robotic RL is also the generation of appropriate reward functions. A vast body of literature investigates how active human feedback can be introduced into an interactive learning loop, with recent publications showing that user feedback can be used for the RL reward. However, increased complexity of robotic skills in everyday environment also increases their dimensionality, which can practically prevent use of user feedback for the reward, because too many trials are needed. In the paper we present the results of using discretized, user-assigned reward for RL of robotic throwing, with an emphasis on learning in the feature space, i. e., latent space of a deep autoencoder network. Statistical evaluation of a user study with 15 participants, who provided feedback for robotic throwing experiments, show that for certain tasks, RL with discrete user feedback can be effectively applied for robot learning.

Attachments: Video Attachment
Keywords: Grasp and motion planning, Exoskeletons and assistive devices
Abstract: Every time a person encounters an object with a given degree of familiarity, he/she immediately knows how to grasp it. Adaptation of the movement of the hand according to the object geometry happens effortlessly because of the accumulated knowledge of previous experiences grasping similar objects. In this paper, we present a novel method for inferring grasp configurations based on the object shape. Grasping knowledge is gathered in a synergy space of the robotic hand built by following a human grasping taxonomy. The synergy space is constructed through human demonstrations employing a exoskeleton that provides force feedback, which provides the advantage of evaluating the quality of the grasp. The shape descriptor is obtained by means of a categorical non-rigid registration that encodes typical intra-class variations. This approach is especially suitable for on-line scenarios where only a portion of the object's surface is observable. This method is demonstrated through simulation and real robot experiments by grasping objects never seen before by the robot.

Keywords: Learning from demonstration, Visual perception, Deep Learning
Abstract: We present a visual imitation learning method that enables robots to imitate demonstrated skills by learning a perceptual reward function based on object-centric feature representations. Our method uses the background configuration of the scene to compute object masks for the objects present. The robotic agent then trains a detector for the relevant objects in the scene via a process we call data dreaming, generating a synthetic dataset of images of various object occlusion configurations using only a small amount of background-subtracted ground truth images. We use the output of the object detector to learn an object-centric visual feature representation. We show that the resulting factorized feature representation comprised of per-object appearance features and cross-object relative locations enables efficient real world reinforcement learning that can teach a robot a policy based on a single demonstration after few minutes of training.

Attachments: Video Attachment
Keywords: Sensorimotor learning, Cognitive development, Deep Learning
Abstract: We propose a tool-use model that can select tools that require neither labeling nor modeling of the environment and actions. With this model, a robot can choose a tool by itself and perform the operation that matches a human command and the environmental situation. To realize the model, we allowed the robot to experience selecting a tool and using it, and we recorded sensory motor data during that time. In addition, we gave the robot task commands, which were integrated with sensory motor data and learned by the model. The experience includes two types of selection, namely according to function and according to size, thereby allowing the robot to handle both situations. For evaluation, the robot is required to generate motion either in an untrained situation or using an untrained tool. We conrm that the robot can choose and use a tool that is suitable for achieving the target task.