Anjum48

Anjum48 t1_j4v8mpm wrote

+1 for UNets. Since IR will be a single channel you could use a single class semantic segmentation-type model (i.e. a UNet with a 1-channel output passed through a sigmoid). Something like this would get you started:

model = sm.Unet('resnet34', classes=1, activation='sigmoid') 

Edit: Forgot the link for the package I'm referencing: https://github.com/qubvel/segmentation_models

Many of the most popular encoders/backbones are implemented in that package

Edit 2: Is the FOV important? If you could resize the images so that the RGB & IR FOV are equivalent then that would make things a lot simpler

4

Anjum48 t1_j40gm5q wrote

Ah ok. On the first point I guess whoever you are looking for will need to spend a considerable amount of time building/finding a dataset to train a model.

On the second point, I might have incorrectly assumed you were familiar with the Zillow controversy around price prediction.

The TL;DR is that the ML team used a model to forecast prices using a tool made by Facebook called Prophet. The model was probably accurate enough for displaying a rough prediction on a website. Another team in Zillow started using these price predictions to flip houses and lost a whole bunch of money since the model was not designed to do this.

A lot of armchair data scientists quickly pointed the finger at Prophet for being a "bad" model. The reality is all models are bad if they are used for the wrong reason. In this case, the team flipping houses likely didn't listen to the data science team when they said the model shouldn't be used for that purpose.

This is why it's a good idea to know how the model outputs are going to be used. The obvious answer is always "as accurate as possible" but sometimes that might not be accurate enough...

Hope this helps!

4

Anjum48 t1_j3lgduo wrote

Are you using the "en_core_web_trf" model in Spacy which is based on the roberta-base transformer model?

If that model is still not accurate enough, you may need to look into using the Hugging Face transformers library and try some more recent transformer models, e.g. deberta

2