We develop deep-learning-based medical-specific models for data from multiple medical domains, such as X-ray, CT image, and Electronic Health record (EMR). The tasks involve disease/organ classification, lesion detection, keypoint extraction, and organ segmentation. Our goal is to effectively adopt state-of-the-art deep learning techniques to medical domain studies.