Electrical Engineering and Systems Science > Image and Video Processing
[Submitted on 30 Apr 2020]
Title:Non-line-of-sight imaging off a Phong surface through deep learning
View PDFAbstract:A deep learning based non-line-of-sight (NLOS) imaging system is developed to image an occluded object off a scattering surface. The neural net is trained using only handwritten digits, and yet exhibits capability to reconstruct patterns distinct from the training set, including physical objects. It can also reconstruct a cartoon video from its scattering patterns in real time, demonstrating the robustness and generalization capability of the deep learning based approach. Several scattering surfaces with varying degree of Lambertian and specular contributions were examined experimentally; it is found that for a Lambertian surface the structural similarity index (SSIM) of reconstructed images is about 0.63, while the SSIM obtained from a scattering surface possessing a specular component can be as high as 0.93. A forward model of light transport was developed based on the Phong scattering model. Scattering patterns from Phong surfaces with different degrees of specular contribution were numerically simulated. It is found that a specular contribution of as small as 5% can enhance the SSIM from 0.83 to 0.93, consistent with the results from experimental data. Singular value spectra of the underlying transfer matrix were calculated for various Phong surfaces. As the weight and the shininess factor increase, i.e., the specular contribution increases, the singular value spectrum broadens and the 50-dB bandwidth is increased by more than 4X with a 10% specular contribution, which indicates that at the presence of even a small amount of specular contribution the NLOS measurement can retain significantly more singular value components, leading to higher reconstruction fidelity. With an ordinary camera and incoherent light source, this work enables a low-cost, real-time NLOS imaging system without the need of an explicit physical model of the underlying light transport process.
Current browse context:
eess.IV
Change to browse by:
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.