Home
:
Book details
:
Book description
Description of
The Complete Course Of Apache Beam 2024
Published 4/2024 MP4 | Video: h264, 1920x1080 | Audio: AAC, 44.1 KHz Language: English | Size: 3.47 GB | Duration: 6h 53m Learn Apache Beam in a Professional way from Scratch. Become an expert in Google Cloud Dataflow, Data Pipeline... What you'll learn At the end of the course you will fully master Apache Beam to be able to define and execute data processing pipelines and dataflow from scratch You will be able to conduct Apache Beam projects step by step, understanding all the logic and ending with advanced practical examples and complete projects You will Understand Apache Beam fundamentals, including batch and stream processing, and learn to install necessary software You will Grasp the core concepts of Apache Beam's programming model, including pipelines, PCollection, and PTransforms, along with windowing You will Acquire practical skills in creating and manipulating pipelines, including reading/writing data and applying transformations You will Master advanced techniques for windowing data and handling event time processing efficiently You will Learn strategies for ensuring data durability, managing state, and handling errors within Apache Beam pipelines You will deeper into advanced concepts such as side inputs, user-defined functions, and dynamic processing within pipelines You will Gain proficiency in testing, debugging, and optimizing Apache Beam pipelines for performance and reliability You will Understand the importance of data encoding, serialization, and type safety in Apache Beam, including strategies for persistence and versioning You will Learn to set up Apache Beam projects in distributed environments like Hadoop, including project configuration in Intellij You will be able to practice the content learned in a practical way by following all the steps in the complete exercises and the hands-on projects Requirements Having basic notions of data processing concepts can help, but the course starts from scratch A decent computer and of course, desire to learn! Description Overview Section 1: Introduction to Apache Beam Lecture 1 Apache Beam Opener Lecture 2 What is Apache Beam? Lecture 3 Batch and stream processing Lecture 4 Installation of Necessary softwares-1 Lecture 5 Installation of Necessary softwares-2 Section 2: Apache Beam Programming Model Lecture 6 Pipelines, PCollection, and PTransforms Lecture 7 Windowing and triggers Lecture 8 Watermarks and event time processing Section 3: Writing Apache Beam Pipelines Lecture 9 Creating a pipeline, Reading and writing data sources Lecture 10 Transformations and ParDo functions Lecture 11 Quiz Section 4: Windowing and Time-based Processing Lecture 12 Windowing strategies. Lecture 13 Handling event time data Lecture 14 Triggers for processing windows Lecture 15 Quiz Section 5: Handling Fault-Tolerance Lecture 16 Data durability and recovery & Managing state in pipelines Lecture 17 Handling errors and exceptions Lecture 18 Quiz Section 6: Advanced Apache Beam Concepts Lecture 19 Side inputs Lecture 20 User-defined functions Lecture 21 Dynamic processing Lecture 22 Quiz Section 7: Testing and Debugging Apache Beam Pipelines Lecture 23 Unit testing Lecture 24 Debugging techniques Lecture 25 Best Practices and Performance Optimization Section 8: Data encoding and type safety Lecture 26 Serialization and deserialization Lecture 27 Bytestream Lecture 28 Persistence Lecture 29 Deepcopy Lecture 30 Versioning Section 9: Apache Beam in Distributed system (Hadoop) Lecture 31 Project Setup in Intellij Lecture 32 Project Setup in Hadoop-1 Lecture 33 Project Setup in Hadoop-2 Section 10: Conclusion and Final Quiz Lecture 34 Final Quiz Lecture 35 Apache Beam Closure