Data Engineering is the plumbing of data science. Almost invisible, but super important and a big mess when done wrong. I Talk about trends, tools and techniques Data Engineering. I want to help you get started and inspire you to create and learn Data Engineering. Check out my Data Engineering Academy and Coaching program. Main Podcast is on YouTube.
#86 The Ultimate Data Engineering Introduction
1:14:35The Podcast is back!!!! I promise I am going to keep it up to date this time ;) In this episode I talk about my newest Data Engineering course. I think it's the ultimate 1 hour 15 minutes introduction to Data Engineering. There were also a ton of questions from the chat that I answered. Think you really enjoy this.
#085 Big Data and Data Science Landscape plus trying to read Tweets with Nifi
43:06We are looking into the network communication protocol map. I first saw this like 10 years ago and its awesome. Then we check out the Big Data and Data Science Landscape image. It shows you all the tools available to do data science, machine learning and data engineering. Which is very helpful if you are researching for tools to use. Before using the Twitter API you got to create a developer account. So, I show you how I created one. After that I tried to get Nifi to download Tweets but it is not working.
#084 Behind the scenes: Audio podcast, free transcriptions and GitHub
51:21Today's podcast is a bit of a behind the scenes. What it takes to do a audio podcast. How you can get audio to text transcriptions for free. .Also Github questions on how to work with branches on the Cookbook
#083 Data Engineering at OLX Case Study
1:10:53Today a case study about OLX with a guest it was super fun! Here are the slides Alexeyand I talked about: https://www.slideshare.net/mobile/AlexeyGrigorev/image-models-infrastructure-at-olx
#082 Reading Tweets With Apache Nifi & IaaS vs PaaS vs SaaS
1:19:06In this episode we install the Nifi docker container and look into how we can extract the twitter data. We are also talking about the differences between infrastructure as a service, platform as a service and application as a service.
#081 How to get tweets from the Twitter API
1:09:47In this episode we look into the Twitter API documentation, which I love by the way. How can we get old tweets for a certain hashtags and how to get current live tweets for these hashtags.
#080 How To Find A Job In Germany & Answering Mails
54:54Tips on how you find a job in Germany and two super interesting mails.
#079 Trying to stay true to myself and making the cookbook public on GitHub
24:34The cookbook my Youtube, it will be for free, forever! Check out the data engineering cookbook on GitHub: https://github.com/andkret/Cookbook
#078 Cookbook collaboration and updates
31:08Updates of the cookbook and how to collaborate on it
#077 Lambda and Kappa Architecture
1:22:01In this episode we talk about the lambda architecture with stream and batch processing as well as a alternative the Kappa Architecture that consists only of streaming. Also Data engineer vs data scientist and we discuss Andrew Ng's AI Transformation Playbook