Previous research on examining relationships between travel behavior and the neighborhood characteristics have shown that number of physical attributes, including dense, well-connected, and diverse environment affect people’s travel behavior, in particular, walking and public transit. Increasing elderly population in most industrialized nations, this study aims to understand travel mode choice of elderly associated with the bulit environment. Unlike previous studies, this study systematically compare influence of walkability in orign and destination of each trip on travel mode choice. Travel distance, purpose and other socio-demographic factors were controlled in the anlaysis. Multilevel fixed effect logistic models show that elderly are more likely to walk and use public transportation than non-elderly residing in similar wakability level. In both elderly and non-elderly groups, influence of walkability in origin is more apparent than influence of walkability in destination on travel mode choice.