https://github.com/jayliu0313/Shape-Guided

Training ⇒ memory bank

1. For each SDF (point cloud patch), we trace back its 500 sampled points (i.e., the input to PointNet) in the 3D receptive field and then calculate their 2D coordinates to retrieve the corresponding RGB features: around 40 to 60 RGB feature vectors. image patches? 好大的MA!!!
2. PointNet is trained with all the anomaly-free samples across category.
3. pretrained resnet 这个太弱了 ⇒ DINO feature

To account for the boundaries of possible defects of holes or cracks, the original mapping is extended by a 2-pixel neighborhood (in blue) to accommodate more RGB features.

Inference

1. SDF score map generation

   1. given query SDF feature F of a patch.

   2. find F’s k nearest features {Fk} in Ms, k=10

      1. k=1, without sparse coding is tested in ablation study
      2. for each feature Fk ⇒ a set of image feature in MA
      3. all the related image feature set: {Ik}, |{Ik}|= 400—600, (k*40—60)

   3. reconstruct F by sparse coding of {Fk} ⇒ F’

      1. from sklearn.decomposition import sparse_encode
      2. sparse有多少帮助？a little see Ablation.

   4. score map = distance between SDF of F and SDF of F’

      

      1. local patches ⇒ feature ⇒ SDF
      2. NIF is patch based and category-agnostic
      3. use all the training patches to train the PointNet and the NIF model
      4. SDF of each point in the patch
2. RGB score map generation
   1. shape guided: for each query image feature, find its 5-nearest features in {Ik}
   2. else steps are similar with SDF score map except NIF model.
3. Score-map alignment
   1. Ssdf + SA(Srgb)
   2. how to learn the linear mapping SA()?
      1. 25 randomly selected training samples =》a set of RGB score maps ⇒ a linear mapping SA(Srgb) such that mean ± 3 × standard deviation of the RGB score distribution would map to their SDF counterpart Ssdf.

Experiments

tab 1, 2

tab 3 还是比较慢, 2s多

fig 10