AI Breakdown

CVPR 2023 - Efficient Scale-Invariant Generator with Column-Row Entangled Pixel Synthesis

In this episode we discuss Efficient Scale-Invariant Generator with Column-Row Entangled Pixel Synthesis
by Thuan Hoang Nguyen, Thanh Van Le, Anh Tran. The paper proposes a new generative model called Column-Row Entangled Pixel Synthesis (CREPS) that can efficiently and scalably synthesize photo-realistic images of any arbitrary resolution. Existing GAN-based solutions suffer from inconsistency and texture sticking issues when scaling output resolution, while INR-based generators have a huge memory footprint and slow inference, making them unsuitable for large-scale or real-time systems. CREPS avoids these problems by using a novel bi-line representation that decomposes layer-wise feature maps into separate "thick" column and row encodings, enabling it to synthesize scale-consistent and alias-free images at any resolution with proper training and inference speed.

CVPR 2023 - IMP: Iterative Matching and Pose Estimation with Adaptive Pooling

In this episode we discuss IMP: Iterative Matching and Pose Estimation with Adaptive Pooling
by Fei Xue, Ignas Budvytis, Roberto Cipolla. The paper proposes an iterative matching and pose estimation framework (IMP) that leverages the geometric connections between the two tasks. They introduce a geometry-aware recurrent attention-based module which jointly outputs sparse matches and camera poses. They also introduce an efficient version of IMP called EIMP, that dynamically discards keypoints without potential matches, reducing the quadratic time complexity of attention computation. The proposed method outperforms previous approaches in terms of accuracy and efficiency on YFCC100m, Scannet, and Aachen Day-Night datasets.

CVPR 2023 - Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection

In this episode we discuss Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection
by Long Li, Junwei Han, Ni Zhang, Nian Liu, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan. The paper proposes a Discriminative co-saliency and background Mining Transformer (DMT) framework for co-salient object detection. The framework includes several economical multi-grained correlation modules that explicitly mine both co-saliency and background information to effectively model their discrimination. These modules include a region-to-region correlation module, contrast-induced pixel-to-token correlation, and co-saliency token-to-token correlation modules. The proposed framework is experimentally validated on three benchmark datasets and the source code is available on GitHub.

CVPR 2023 - ABLE-NeRF: Attention-Based Rendering with Learnable Embeddings for Neural Radiance Field

In this episode we discuss ABLE-NeRF: Attention-Based Rendering with Learnable Embeddings for Neural Radiance Field
by Zhe Jun Tang, Tat-Jen Cham, Haiyu Zhao. The paper presents an alternative approach to the Neural Radiance Field (NeRF) method for representing 3D scenes that addresses view-dependent effects such as murky glossy and translucent surfaces. The proposed method, called ABLE-NeRF, uses a self-attention-based framework on volumes along a ray and incorporates Learnable Embeddings to capture view-dependent effects. The results show that ABLE-NeRF significantly reduces blurry glossy surfaces and produces realistic translucent surfaces, surpassing Ref-NeRF in all three image quality metrics.

CVPR 2023 - A Dynamic Multi-Scale Voxel Flow Network for Video Prediction

In this episode we discuss A Dynamic Multi-Scale Voxel Flow Network for Video Prediction
by Xiaotao Hu, Zhewei Huang, Ailin Huang, Jun Xu, Shuchang Zhou. The paper proposes a Dynamic Multi-scale Voxel Flow Network (DMVFN) for video prediction using only RGB images. The proposed network is efficient and achieves better performance than previous methods that require extra inputs for promising performance. The core of DMVFN is a differentiable routing module that effectively perceives the motion scales of video frames and selects adaptive sub-networks for different inputs at the inference stage. DMVFN outperforms state-of-the-art iterative-based OPT on generated image quality and is an order of magnitude faster than Deep Voxel Flow.

CVPR 2023 - Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel

In this episode we discuss Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel
by Xudong Lin, Simran Tiwari, Shiyuan Huang, Manling Li, Mike Zheng Shou, Heng Ji, Shih-Fu Chang. The paper discusses the challenge of multi-channel video-language retrieval, which requires models to understand information from different sources such as video and text. The authors investigate different options for representing videos and fusing video and text information using a principled model design space. The evaluation of four combinations on five video-language datasets reveals that discrete text tokens with a pretrained contrastive text model perform the best, even outperforming state-of-the-art models on some datasets. The authors attribute this to the ability of text tokens to capture key visual information and align naturally with strong text retrieval models.

CVPR 2023 - Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Images

In this episode we discuss Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Images
by Tiancheng Lin, Zhimiao Yu, Hongyu Hu, Yi Xu, Chang Wen Chen. The paper proposes a new scheme called Interventional Bag Multi-Instance Learning (IBMIL) to improve the classification of whole slide pathological images. Existing methods focus on improving feature extraction and aggregation but may capture spurious correlations between bags and labels. IBMIL uses backdoor adjustment for interventional training to suppress bias caused by contextual priors and achieves consistent performance boosts, making it a state-of-the-art method. Code for IBMIL is available on GitHub.

CVPR 2023 - Devil is in the Queries: Advancing Mask Transformers for Real-world Medical

In this episode we discuss Devil is in the Queries: Advancing Mask Transformers for Real-world Medical
by Mingze Yuan, Yingda Xia, Hexin Dong, Zifan Chen, Jiawen Yao, Mingyan Qiu, Ke Yan, Xiaoli Yin, Yu Shi, Xin Chen, Zaiyi Liu, Bin Dong, Jingren Zhou, Le Lu, Ling Zhang, Li Zhang. The paper proposes a method for medical image segmentation that is capable of accurately identifying rare and clinically significant conditions, known as tail conditions. The method utilizes object queries in Mask Transformers to assign soft clusters during training and detect out-of-distribution (OOD) regions during inference, which is referred to as MaxQuery. The authors also introduce a query-distribution (QD) loss to improve segmentation of inliers and OOD indication. The proposed framework outperforms previous state-of-the-art algorithms on pancreatic and liver tumor segmentation tasks.

CVPR 2023 - Inverting the Imaging Process by Learning an Implicit Camera Model

In this episode we discuss Inverting the Imaging Process by Learning an Implicit Camera Model
by Xin Huang, Qi Zhang, Ying Feng, Hongdong Li, Qing Wang. The paper introduces a new approach for modeling the physical imaging process of a camera as an implicit neural network, which is able to learn and control camera parameters. This approach is tested on two challenging inverse imaging tasks: all-in-focus and HDR imaging. The results show that the new implicit neural camera model is able to produce visually appealing and accurate images, making it a promising tool for a wide range of inverse imaging tasks.

CVPR 2023 - Label-Free Liver Tumor Segmentation

In this episode we discuss Label-Free Liver Tumor Segmentation
by Qixin Hu, Yixiong Chen, Junfei Xiao, Shuwen Sun, Jieneng Chen, Alan Yuille, Zongwei Zhou. The paper discusses the use of synthetic tumors in CT scans to train AI models to accurately segment liver tumors without the need for manual annotation. These synthetic tumors are realistic in shape and texture and have proven effective in training the AI models, which demonstrated similar performance to models trained on real tumors. This highlights the potential for significantly reducing manual efforts for tumor annotation and the ability to improve the success rate of detecting small liver tumors, while also allowing for rigorous assessment of AI robustness.