Related Researcher

Author's Photo

Ahn, Hyemin
AI & Human-Robot Interaction Lab.
Research Interests

Text2Action: Generative Adversarial Synthesis from Language to Action

DC Field Value Language Ahn, Hyemin ko Ha, Timothy ko Choi, Yunho ko Yoo, Hwiyeon ko Oh, Songhwai ko 2022-07-15T00:57:46Z - 2022-06-08 ko 2018-05-21 ko
dc.identifier.citation IEEE International Conference on Robotics and Automation, pp.5915 - 5920 ko
dc.identifier.issn 1050-4729 ko
dc.identifier.uri -
dc.description.abstract In this paper, we propose a generative model which learns the relationship between language and human action in order to generate a human action sequence given a sentence describing human behavior. The proposed generative model is a generative adversarial network (GAN), which is based on the sequence to sequence (SEQ2SEQ) model. Using the proposed generative network, we can synthesize various actions for a robot or a virtual agent using a text encoder recurrent neural network (RNN) and an action decoder RNN. The proposed generative network is trained from 29,770 pairs of actions and sentence annotations extracted from MSR-Video-to-Text (MSR-VTT), a large-scale video dataset. We demonstrate that the network can generate human-like actions which can be transferred to a Baxter robot, such that the robot performs an action based on a provided sentence. Results show that the proposed generative network correctly models the relationship between language and action and can generate a diverse set of actions from the same sentence. ko
dc.language 영어 ko
dc.publisher IEEE COMPUTER SOC ko
dc.title Text2Action: Generative Adversarial Synthesis from Language to Action ko
dc.type CONFERENCE ko
dc.identifier.scopusid 2-s2.0-85055121346 ko
dc.identifier.wosid 000446394504069 ko
dc.type.rims CONF ko
dc.identifier.doi 10.1109/ICRA.2018.8460608 ko
dc.identifier.url ko
Appears in Collections:
AI_Conference Papers

find_unist can give you direct access to the published full text of this article. (UNISTARs only)

Show simple item record


  • mendeley


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.