Index

As this ebook edition doesn't have fixed pagination, the page numbers below are hyperlinked for reference only, based on the printed edition of this book.

Symbols

3D coordination systems 22-24

3D data file format

OBJ files format 13-22

Ply files format 7-13

3D data representation 5

mesh representation 6

point cloud representation 5

voxel representation 6, 7

3D human pose and shape

estimating, with SMPLify 160

3D modeling problem

formulating 152

representation, defining 153, 154

3D models

reconstructing, from multi-view images 104-109

3D rendering

coding exercises 34-41

A

Anaconda

URL 4

B

barycentric coordinates 31, 32

blend shapes 155

C

camera models 24, 25

coding 25-28

Chamfer distance 57

CMU Graphics Lab Motion Capture Database 161

compositional 3D-aware image synthesis 135-138

compositional operator 136

controllable car generation

exploring 142, 143

controllable face generation

exploring 144-146

controllable scene generation

exploring 141

Convolutional Neural Network (CNN) 136

coordination systems

coding 25-28

cost function 57

D

deformable mesh fitting problem

formulating, into optimization problem 56, 57

development environment

setting up 4, 5

differentiable rendering

creating 70-72

need for 68-70

used, for solving issues 72

differentiable volume rendering 104

3D models, reconstructing from multi-view images 104-109

E

exponential map 48

F

feature field 139

generating 138

mapping, to images 139, 141

Frechet Inception Distance 147

G

GAN-based image synthesis 134, 135

GAN model architecture 136

Generative Adversarial Networks (GANs) 134

GIRAFFE model 135

compositional operator 136

Frechet Inception Distance 147

GAN model architecture 136

learning 3D representation 136

neural rendering model 136

pre-trained dataset models 141

training 146, 147

GIRAFFE model, training parameters

data 148

model 148

training 148

graph convolutional networks (GCNs) 194

graph convolutions 194, 195

H

hat operator 48

heterogeneous mini-batch

coding exercise 43-47

list format 44

packed format 44

padded format 44

I

identity-based shape 156

images

feature field, mapping to 139, 141

L

Lambertian shading model 33

Laplacian 58

learning 3D representation 136

light source models 32

Linear Blend Skinning technique 154, 155

logarithmic map 48

loss functions 55

properties requisites 55

M

Material Template Library (MTL) file 13

mesh

overview 190, 191

mesh edge loss 59

usage, experimenting 64

meshes, fitting to point clouds

problem 55

mesh fitting

implementing, with PyTorch3D 59-63

mesh Laplacian smoothing loss 58

mesh normal consistency loss 58

mesh predictor 196-198

Mesh R-CNN

using, with PyTorch 198-206

Mesh R-CNN architecture 191-194

graph convolutions 194, 195

mesh predictor 196-198

mesh representation 6

Multi-Layer Perceptron (MLP) 114

multi-view images

3D models, reconstructing from 104-109

N

NeRF model 114

architecture 123-128

training 114-123

neural networks

used, for representing radiance fields 113

neural point cloud renderer

versus naïve renderers 173

Neural Radiance Fields (NeRF) 111-133

neural rendering model 136

Normalized Device Coordinates (NDCs) 117

O

object pose estimation

code 76-84

example, for silhouette fitting 84-93

example, for texture fitting 84-93

problem 72-75

OBJ files format 13-22

optimization objective function

defining 160, 161

optimization problem

deformable mesh fitting problem, formulating into 56, 57

P

Phong lighting model 33, 34

Ply files format 7-13

point cloud representation 5

PointNet

URL 5

PyTorch

Mesh R-CNN, using with 198-206

PyTorch3D

used, for implementing mesh fitting 59-63

PyTorch3D heterogeneous batches

using 42, 43

PyTorch optimizers

using 42, 43

R

radiance field 112, 113

representing, with neural networks 113

volume rendering 128

ray marcher

exploring 102, 103

ray marching 97

rays

color, accumulating 129, 130

projecting, into scene 129

ray sampling 96-100

region proposal network (RPN) 194

regularization 58

regularization loss functions 57

mesh edge loss 59

mesh Laplacian smoothing loss 58

mesh normal consistency loss 58

mesh, obtained with mesh edge loss 64

mesh, without regularization loss functions 63

rendering 30, 31

representation

defining 153, 154

rotations 47, 48

coding exercise 49, 50

S

scene

rays, projecting into 129

shape 156

shape-based shape 156

Signed Distance Function (SDF) 6

silhouette fitting

object pose estimation, example 84-93

Skinned Multi-Person Linear (SMPL) model 156

defining 156

pose-dependent template mesh 157

shape dependent joints 157

using 157-159

skinning 154

SMPLify

code, exploring 164-167

code, running 162, 163

exploring 161

stages 160

used, for estimating 3D human pose and shape 160

stereo vision cameras 54

Stochastic Gradient Descent (SGD) 42

Structural Similarity Index (SSIM) 179

SynSin model

implementing 176-186

network architecture 171

SynSin network architecture 171

depth network 172

discriminator 176

neural point cloud renderer 172-175

refinement module 175, 176

spatial feature 172

T

texture fitting

object pose estimation, example 84-93

time-of-flight cameras 54

transformations 47, 48

coding exercise 49, 50

Truncated Signed Distance Functions (TSDFs) 6

V

view synthesis 112

challenges 170

methods 171

overview 170, 171

volume 96

volume (volumetric) rendering 128

overview 96, 97

with radiance fields 128

volume sampling 97, 128

using 100-102

voxel representation 6, 7

W

world coordination system 22