Melanoma detection using a smartphone application

In early 2017, we started a project to develop a smartphone application to automatically detect melanoma. This project was the Gabriel Ucelli’s undergraduate thesis, which you can check in this link (unfortunately, it’s in Portuguese only). The Prof. Renato A. Krohling was the advisor and I was the co-advisor of this work.

As we didn’t have our own dataset, we used a public one provided by ISIC. This dataset is composed of dermoscopy images. You may see some samples below:

The project’s goal was quite simple: designing a deep lerning model to detect melanoma and embed it in a smartphone application. We tried different CNN models in order to solve this problem. We figured out that very deep models, like VGG, inception V3, GoogleNet, were not good options because the network may consume all smartphone memory. In addition, for this kind of network, the connection weights become a large file to save/load, which is not good for a smartphone as well. Thus, the best option we found was the <a href=“https://ai.googleblog.com/2017/06/mobilenets-open-source-models-for.html” target=“_blank” rel=“noopener”>mobileNet</a> [1], a CNN developed specially to work properly on smartphones. The mobileNet’s trained model achieved the following performance:

Next, we deployed this model in the smartphone application. All trained models and discussion are described in the thesis.

The application was developed using Java and Kotlin. You can see some screenshots below:

First, we need to crop a region on the skin. Next, the app shows the probability to be a melanoma or others. In this case, this spot on my skin has 99,5% of chance, according to the model, to be others.

Nonetheless, we have a problem here. The network was trained using dermoscopy images and we used the smartphone’s camera to take the pic. In this case specifically, it works well. However, it’s not the usual. The solution is to train the network using smartphone images. But, where can we find these images? Well, as far as we know, there’s no public dataset for that. Thus, we started a new project to fulfill this gap.

In any case, this app was a proof of concept. It works well for dermoscopy images. It was developed only for androids and you can download the .apk by clicking in this link. The source code is also available on https://github.com/gabrielucelli/Melanoma-Detector

However, it is important to note that we do not intend to release such an app for general users. As mentioned, it was just a proof of concept. Our goal now is to help clinicians to improve the skin cancer diagnosis in particular in Brazil’s countryside.

[1] Howard, Andrew G., et al. “Mobilenets: Efficient convolutional neural networks for mobile vision applications.” arXiv preprint arXiv:1704.04861(2017).