Jaundice is a yellow discoloration of the skin, mucous membranes, and sclera caused by increased amounts of total bilirubin in the blood. Patients frequently visiting to the hospital to monitor jaundice and measuring the total bilirubin values are uncomfortable. To solve these problems, this paper proposes an image-based diagnostic method of jaundice for mobile devices. The proposed algorithm extracts the scleral region by applying the Otsu method to the patient image captured by mobile devices, performs preprocessing to correct image and removes the light reflection in the scleral region. Then the proposed algorithm based on the HSV color model is used to obtain image feature values that show the progression of jaundice. Simulation results have been obtained by applying the proposed method to image history of jaundice patients, which show that total bilirubin values are well matched with the image feature values.