![]() ![]() This guide discusses the benefits of using an XCom backend, shows an example of implementing an XCom backend with S3, and describes how to set this up if you’re running Airflow on Astro. You can also implement your own serialization / deserialization methods to define how XComs are handled. This means that rather than store XComs in Airflow’s metadata database by default, you can push and pull XComs to and from an external system such as S3, GCS, or HDFS. One of the features of the TaskFlow API that increases the flexibility of XComs is support for a custom XCom backend. The TaskFlow API abstracts the task and dependency management layer away from users, which greatly improves the experience of working with XComs. One of the main new features in this release was the TaskFlow API, which was introduced to solve the challenge of explicitly passing messages between Airflow tasks. ![]() In December 2020, the Apache Airflow project took a huge step forward with the release of Airflow 2.0. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |