why the localNet not use global depth prediction as input

[q5390498]

I am confuse that the dense depth is generated, why not use it as localnet's input? Does instead the guidance map to global depth prediction will bad to the accuracy?

[wvangansbeke]

Not sure I understand your question. The ground truth (i.e. "dense depth") can not be used as an input during training. We need to predict it after all. What would you do at test time?

[q5390498]

Thank for reply. The global net has generate global depth prediction, conf weights, guidence map, I mean why you need guidence map as localnet's input? why donot you use global depth prediction as localnet's input directly?

[wvangansbeke]

Hi @q5390498 ,

Ok makes sense. I believe that would work fine. Maybe quickly try it out. However, keep in mind that fusion takes place at multiple stages in the latest version. I remember that early + late fusion was the best strategy.

[q5390498]

Thank you very much. In localnet, we use the sparse depth map A and guidence map B as inputs, suppose we also use A as gt, does it make sense? I know the kitti dataset has more densely gt, but maybe it is hard to make custom dataset. Or can you give me some suggestions？