Ravi Dayabhai & Conrad Warren 🧱 2024-06-21
Consider the following array of 25 squares:
You are filling the array with rectangles by repeating the following two steps:
| from functools import reduce | |
| from transforms.api import transform_df, Input, Output | |
| @transform_df( | |
| Output("/data_project/manifests/trp_synced_paths_manifest"), | |
| incremental_manifest=Output("/data_project/manifests/trp_incremental_manifest"), | |
| synced_paths=Input("/data_project/manifests/trp_synced_paths_manifest"), | |
| bucket_data=Input("/data_project/input/s3_bucket_data_source") # Data source for your bucket content | |
| ) | |
| def incremental_sync_and_update(ctx, synced_paths, bucket_data): |