Computer Vision Decoded

In this episode of Computer Vision Decoded, we bring to you a live recording of Jared Heinly presentation on the evolution of image based 3D reconstruction. This recording was from a Computer Vision Decoded meetup in Pittsburgh with a visual component. If you would like to tune in with the visuals, we recommend watching the episode on our YouTube channel: https://youtu.be/Gwib5IcTKHIFollow:Jared on X: https://x.com/JaredHeinlyJonathan on X: https://x.com/jonstephens85This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services. Learn more at https://www.everypoint.io

In this episode of Computer Vision Decoded, our hosts Jonathan Stephens and Jared Heinly are joined by Ruilong Li, a researcher at NVIDIA and key contributor to both Nerfstudio and gsplat, to dive deep into 3D Gaussian Splatting. They explore how this relatively new technology works, from the fundamentals of gaussian representations to the optimization process that creates photorealistic 3D scenes. Ruilong explains the technical details behind gaussian splatting, and discusses the development of the popular gsplat library. The conversation covers practical advice for capturing high-quality data, the iterative training process, and how Gaussian splatting compares to other 3D representations like meshes and NeRFs.Links: gsplat: https://github.com/nerfstudio-project/gsplatNerfstudio: https://docs.nerf.studio/Follow:Ruilong on X: https://x.com/ruilong_liJared on X: https://x.com/JaredHeinlyJonathan on X: https://x.com/jonstephens85This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services. Learn more at https://www.everypoint.io

In this episode of Computer Vision Decoded, hosts Jonathan Stephens and Jared Heinly explore the various types of cameras used in computer vision and 3D reconstruction. They discuss the strengths and weaknesses of smartphone cameras, DSLR and mirrorless cameras, action cameras, drones, and specialized cameras like 360, thermal, and event cameras. The conversation emphasizes the importance of understanding camera specifications, metadata, and the impact of different lenses on image quality. The hosts also provide practical advice for beginners in 3D reconstruction, encouraging them to start with the cameras they already own.TakeawaysSmartphones are versatile and user-friendly for photography.RAW images preserve more data than JPEGs, aiding in post-processing.Mirrorless and DSLR cameras offer better low-light performance and lens flexibility.Drones provide unique perspectives and programmable flight paths for capturing images.360 cameras allow for quick scene capture but may require additional processing for 3D reconstruction.Event cameras capture rapid changes in intensity, useful for robotics applications.Thermal and multispectral cameras are specialized for specific applications, not typically used for 3D reconstruction.Understanding camera metadata is crucial for effective image processing.Choosing the right camera depends on the specific needs of the project.Starting with a smartphone is a low barrier to entry for beginners in 3D reconstruction.This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode, Jonathan Stephens and Jared Heinly delve into the intricacies of COLMAP, a powerful tool for 3D reconstruction from images. They discuss the workflow of COLMAP, including feature extraction, correspondence search, incremental reconstruction, and the importance of camera models. The conversation also covers advanced topics like geometric verification, bundle adjustment, and the newer GLOMAP method, which offers a faster alternative to traditional reconstruction techniques. Listeners are encouraged to experiment with COLMAP and learn through hands-on experience.This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode, we discuss practical tips and challenges in 3D reconstruction from images, focusing on various environments such as urban, indoor, and outdoor settings. We explore issues like repetitive structures, lighting conditions, and the impact of reflections and shadows on reconstruction quality. The conversation also touches on the importance of camera motion, lens distortion, and the role of machine learning in enhancing reconstruction processes. Listeners gain insights into optimizing their 3D capture techniques for better results.Key TakeawaysRepetitive structures can confuse computer vision algorithms.Lighting conditions greatly affect image quality and reconstruction accuracy.Wide-angle lenses can help capture more unique features.Indoor environments present unique challenges like textureless walls.Aerial imaging requires careful management of lens distortion.Understanding the application context is crucial for effective 3D reconstruction.Camera motion should be varied to avoid distortion and drift.Planning captures based on goals can lead to better results.This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services. Learn more at https://www.everypoint.io

In this episode of Computer Vision Decoded, Jonathan Stephens and Jared Heinly explore the concept of depth maps in computer vision. They discuss the basics of depth and depth maps, their applications in smartphones, and the various types of depth maps. The conversation delves into the role of depth maps in photogrammetry and 3D reconstruction, as well as future trends in depth sensing and machine learning. The episode highlights the importance of depth maps in enhancing photography, gaming, and autonomous systems.Key Takeaways:Depth maps represent how far away objects are from a sensor.Smartphones use depth maps for features like portrait mode.There are multiple types of depth maps, including absolute and relative.Depth maps are essential in photogrammetry for creating 3D models.Machine learning is increasingly used for depth estimation.Depth maps can be generated from various sensors, including LiDAR.The resolution and baseline of cameras affect depth perception.Depth maps are used in gaming for rendering and performance optimization.Sensor fusion combines data from multiple sources for better accuracy.The future of depth sensing will likely involve more machine learning applications.Episode Chapters00:00 Introduction to Depth Maps00:13 Understanding Depth in Computer Vision06:52 Applications of Depth Maps in Photography07:53 Types of Depth Maps Created by Smartphones08:31 Depth Measurement Techniques16:00 Machine Learning and Depth Estimation19:18 Absolute vs Relative Depth Maps23:14 Disparity Maps and Depth Ordering26:53 Depth Maps in Graphics and Gaming31:24 Depth Maps in Photogrammetry34:12 Utilizing Depth Maps in 3D Reconstruction37:51 Sensor Fusion and SLAM Technologies41:31 Future Trends in Depth Sensing46:37 Innovations in Computational PhotographyThis episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services. Learn more at https://www.everypoint.io

After an 18 month hiatus, we are back! In this episode of Computer Vision Decoded, hosts Jonathan Stephens and Jared Heinly discuss the latest advancements in computer vision technology, personal updates, and insights from the industry. They explore topics such as real-time 3D reconstruction, computer vision research, SLAM, event cameras, and the impact of generative AI on robotics. The conversation highlights the importance of merging traditional techniques with modern machine learning approaches to solve real-world problems effectively.Chapters00:00 Intro & Personal Updates04:36 Real-Time 3D Reconstruction on iPhones09:40 Advancements in SfM14:56 Event Cameras17:39 Neural Networks in 3D Reconstruction26:30 SLAM and Machine Learning Innovation29:48 Applications of SLAM in Robotics34:19 NVIDIA's Cosmos and Physical AI40:18 Generative AI for Real-World Applications43:50 The Future of Gaussian Splatting and 3D ReconstructionThis episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode of Computer Vision Decoded, we are going to dive into our in-house computer vision expert's reaction to the iPhone 15 and iPhone 15 Pro announcement.We dive into the camera upgrades, decode what a quad sensor means, and even talk about the importance of depth maps.Episode timeline:00:00 Intro02:59 iPhone 15 Overview05:15 iPhone 15 Main Camera07:20 Quad Pixel Sensor Explained15:45 Depth Maps Explained22:57 iPhone 15 Pro Overview27:01 iPhone 15 Pro Cameras32:20 Spatial Video36:00 A17 Pro ChipsetThis episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode of Computer Vision Decoded, we are going to dive into Pierre Moulon's 10 years experience building OpenMVG. We also cover the impact of open-source software in the computer vision industry and everything involved in building your own project. There is a lot to learn here!Our episode guest, Pierre Moulon, is a computer vision research scientist and creator of OpenMVG -  a library for computer-vision scientists and targeted for the Multiple View Geometry community.The episode follow's Pierre's journey building OpenMVG which he wrote about as an article in his GitHub repository.Explore OpenMVG on GitHub: https://github.com/openMVG/openMVG Pierre's article on building OpenMVG: https://github.com/openMVG/openMVG/discussions/2165Episode timeline:00:00 Intro01:00 Pierre Moulon's Background04:40 What is OpenMVG?08:43 What is the importance of open-source software for the computer vision community?12:30 What to look for deciding to use an opensource project16:27 What is Multi View Geometry?24:24 What was the biggest challenge building OpenMVG?31:00 How do you grow a community around an open-source project38:09 Choosing a licensing model for your open-source project43:07 Funding and sponsorship for your open-source project46:46 Building an open-source project for your resume49:53 How to get started with OpenMVGContact:Follow Pierre Moulon on LinkedIn: https://www.linkedin.com/in/pierre-moulon/Follow Jared Heinly on Twitter: https://twitter.com/JaredHeinlyFollow Jonathan Stephens on Twitter at: https://twitter.com/jonstephens85This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode of Computer Vision Decoded, we are going to dive into implicit neural representations.We are joined by Itzik Ben-Shabat, a Visiting Research Fellow at the Australian National Universit (ANU) and Technion – Israel Institute of Technology as well as the host of the Talking Paper Podcast.You will learn a core understanding of implicit neural representations, key concepts and terminology, how it's being used in applications today, and Itzik's research into improving output with limit input data.Episode timeline:00:00 Intro01:23 Overview of what implicit neural representations are04:08 How INR compares and contrasts with a NeRF08:17 Why did Itzik pursued this line of research10:56 What is normalization and what are normals 13:13 Past research people should read to learn about the basics of INR 16:10 What is an implicit representation (without the neural network)24:27 What is DiGS and what problem with INR does it solve?35:54 What is OG-I NR and what problem with INR does it solve?40:43 What software can researchers use to understand INR?49:15 What information should non-scientists be focused to learn about INR?Itzik's Website: https://www.itzikbs.com/Follow Itzik on Twitter: https://twitter.com/sitzikbsFollow Itzik on LinkedIn: https://www.linkedin.com/in/yizhak-itzik-ben-shabat-67b3b1b7/Talking Papers Podcast: https://talking.papers.podcast.itzikbs.com/Follow Jared Heinly on Twitter: https://twitter.com/JaredHeinlyFollow Jonathan Stephens on Twitter at: https://twitter.com/jonstephens85Referenced past episode- What is CVPR: https://share.transistor.fm/s/15edb19dThis episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io