3D cartoon face and cat face from one image

栏目: IT技术 · 发布时间: 4年前

内容简介:Summary: We propose a method to learn weakly symmetric deformable 3D object categories from raw single-view images, without ground-truth 3D, multiple views, 2D/3D keypoints, prior shape models or any other supervision.[We store a copy of the uploaded image

Summary: We propose a method to learn weakly symmetric deformable 3D object categories from raw single-view images, without ground-truth 3D, multiple views, 2D/3D keypoints, prior shape models or any other supervision.

[ Paper · Project Page · Code ]

Demo

Input

Upload your own image (<1MB)

Lighting Mode

Share

We store a copy of the uploaded image for 7 days, after which it will be automatically deleted. The uploaded image is not used for any other purpose.

or select one of the examples below

Method Overview

We propose a method to learn 3D deformable object categories from raw single-view images, without any manual or external supervision . The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to disentangle these components without supervision, we use the fact that many object categories have, at least in principle, a symmetric structure. We show that reasoning about illumination allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, we model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model.

Photo-Geometric Autoencoding

Our method is based on an autoencoder that factors each input image into depth , albedo , viewpoint and lighting . These four components are combined to reconstruct the input image. The model is trained only using a reconstruction loss, without any external supervision.

Exploiting Symmetry

In order to achieve this decomposition without supervision, we exploit the fact that many object categories have a bilateral symmetry . Assuming an object is perfectly symmetric, one can obtain a virtual second view of it by simply mirroring the image and perform 3D reconstruction using stereo geometry [1, 2].

Here, we would like to leverage this symmetry assumption. We enforce the model to predict a symmetric view of the object by injecting a flipping operation, and obtain two reconstructions (with and without flipping) of the same input view through predicted viewpoint transformation. Minimizing two reconstruction losses at the same time essentially imposes a “two-view” constraint and provides sufficient signal for recovering accurate 3D shapes.

Note that even if an object has symmetric intrinsic textures (aka. albedo), it may still result in an asymmetric appearance due to asymmetric illumination. Here, this is handled by predicting albedo and lighting separately, and enforcing symmetry only on albedo while allowing the shading to be asymmetric. We assume a simple Lambertian illumination model, and compute a shading map from the predicted light direction and depth map.

In fact, doing so does not only allow the model to learn accurate intrinsic image decomposition, but also provides strong regularization on the shape prediction (similar to shape from shading)! Unnatural shapes are avoided since they result in unnatural shading and thus a higher reconstruction loss.

Probabilistic Modeling of Symmetry using Confidence Maps

Although symmetry provides strong signal for recovering 3D shapes, specific object instances are in practice never fully symmetric. We account for potential asymmetry using uncertainty modeling [3]. Our model additionally predicts a pair of per-pixel confidence maps, and is trained to minimize the two confidence-adjusted reconstruction losses at the same time, and with asymmetric weights to allow for a dominant side.

References

[1] Mirror Symmetry ⇒ 2-View Stereo Geometry. Alexandre R. J. François, Gérard G. Medioni, and Roman Waupotitsch. Image and Vision Computing, 2003.

[2] Detecting and Reconstructing 3D Mirror Symmetric Objects. Sudipta N. Sinha, Krishnan Ramnath, and Richard Szeliski. Proc. ECCV, 2012.

[3] What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Alex Kendall and Yarin Gal. NeurIPS, 2017.

Author’s webpage: Shangzhe & Christian


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

产品经理面试宝典

产品经理面试宝典

[美] Gayle Laakmann McDowell、[美]Jackie Bavaro / 吴海星、陈少芸 / 人民邮电出版社 / 2015-3 / 59.00元

本书针对IT 行业产品经理,以面试为主线,首先介绍产品经理职责以及谷歌、微软等知名企业中产品经理的作用和要求;然后采访了几位知名企业的产品经理,介绍成为产品经理的基本素质;之后从简历准备、各公司面试要点到具体面试问题进行详细分析,这部分是本书的重点内容。读者对象包括IT 行业产品经理以及对如何做好产品有兴趣的人士。一起来看看 《产品经理面试宝典》 这本书的介绍吧!

HTML 编码/解码
HTML 编码/解码

HTML 编码/解码

XML、JSON 在线转换
XML、JSON 在线转换

在线XML、JSON转换工具

RGB CMYK 转换工具
RGB CMYK 转换工具

RGB CMYK 互转工具