MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images

1KAIST, 2Stanford University 3NVIDIA

MV2Cyl, our novel 3D CAD reverse engineering method, reconstructs a 3D CAD model from multi-view images using NeRF and 2D segmentation modules.

Abstract

We present MV2Cyl, a novel method for reconstructing 3D from 2D multi-view images, not merely as a field or raw geometry but as a sketch-extrude CAD model. Extracting extrusion cylinders from raw 3D geometry has been extensively re- searched in computer vision, while the processing of 3D data through neural networks has remained a bottleneck. Since 3D scans are generally accompanied by multi-view images, leveraging 2D convolutional neural networks allows these images to be exploited as a rich source for extracting extrusion cylinder informa- tion. However, we observe that extracting only the surface information of the extrudes and utilizing it results in suboptimal outcomes due to the challenges in the occlusion and surface segmentation. By synergizing with the extracted base curve information, we achieve the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation. Our experiments, comparing our method with previous work that takes a raw 3D point cloud as input, demonstrate the effectiveness of our approach by taking advantage of multi-view images.


Method


reformulation

MV2Cyl reconstructs 3D extrusion cylinders from multi-view images without relying on raw 3D geometry as input. The idea is to leverage 2D neural networks to learn 2D priors that provide 3D extrusion information, i.e. extrusion curves and surfaces. The integration of the information into 3D by optimizing neural fields. MV2Cyl is a combination of a curve field and a surface field that is used to recover the parameters and reconstruct the extrusion cylinders only given 2D input images.

Qualitative Results from 3D Reconstruction


Real-World Demo


Citation

Please consider citing our work if you find it useful.


      
      @inproceedings{hong2024mvcyl,
        title={{MV}2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images},
        author={Hong, Eunji and Nguyen, Minh Hieu and Uy, Mikaela Angelina and Sung, Minhyuk},
        booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
        year={2024},
      }