Last active
November 6, 2021 22:27
-
-
Save blacha/57114537c2263f77232c76ae56f14d8d to your computer and use it in GitHub Desktop.
Docker GDAL COG creation issues
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| GDAL_CONTAINER=osgeo/gdal:ubuntu-small-latest | |
| TIF_FOLDER= | |
| docker run -v $PWD:$PWD -it $GDAL_CONTAINER gdalbuildvrt \ | |
| $PWD/$TIF_FOLDER.vrt \ | |
| $PWD/$TIF_FOLDER/*.tif | |
| docker run -v $PWD:$PWD -it $GDAL_CONTAINER gdal_translate \ | |
| -of COG \ | |
| -co COMPRESS=WEBP \ | |
| -co NUM_THREADS=ALL_CPUS \ | |
| -co BIGTIFF=YES \ | |
| --config BIGTIFF_OVERVIEW YES \ | |
| --config GDAL_CACHEMAX 8196 \ | |
| $PWD/$TIF_FOLDER.vrt $PWD/$TIF_FOLDER.webp.cog.tif |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Samples: 26K of event 'cpu-clock', Event count (approx.): 260850000000 | |
| Children Self Command Shared Object Symbol | |
| + 69.46% 16.13% gdal_translate ld-musl-x86_64.so.1 [.] memset | |
| + 56.34% 0.00% gdal_translate [kernel.kallsyms] [k] page_fault | |
| + 56.34% 0.02% gdal_translate [kernel.kallsyms] [k] do_page_fault | |
| + 56.17% 24.83% gdal_translate [kernel.kallsyms] [k] __do_page_fault | |
| + 27.35% 1.38% gdal_translate [kernel.kallsyms] [k] handle_mm_fault | |
| + 25.52% 2.50% gdal_translate [kernel.kallsyms] [k] __handle_mm_fault | |
| + 12.14% 0.30% gdal_translate [kernel.kallsyms] [k] alloc_pages_vma | |
| + 11.50% 0.64% gdal_translate [kernel.kallsyms] [k] __alloc_pages_nodemask | |
| + 7.92% 0.00% gdal_translate [kernel.kallsyms] [k] entry_SYSCALL_64_after_hwframe | |
| + 7.92% 0.04% gdal_translate [kernel.kallsyms] [k] do_syscall_64 | |
| + 7.31% 0.02% gdal_translate ld-musl-x86_64.so.1 [.] __munmap | |
| + 7.26% 0.00% gdal_translate [kernel.kallsyms] [k] __x64_sys_munmap | |
| + 7.26% 0.00% gdal_translate [kernel.kallsyms] [k] __vm_munmap | |
| + 7.24% 0.01% gdal_translate [kernel.kallsyms] [k] __do_munmap | |
| + 6.98% 0.00% gdal_translate [kernel.kallsyms] [k] unmap_region | |
| + 6.83% 6.78% gdal_translate [kernel.kallsyms] [k] clear_page_erms | |
| + 6.67% 6.67% gdal_translate [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Samples: 16K of event 'cpu-clock', Event count (approx.): 165400000000 | |
| Children Self Command Shared Object Symbol | |
| + 56.69% 0.00% gdal_translate libc-2.27.so [.] 0x00007f30814dbf2d | |
| + 55.94% 55.94% gdal_translate libc-2.27.so [.] 0x000000000018ef2d | |
| + 6.58% 0.00% gdal_translate [kernel.kallsyms] [k] page_fault | |
| + 6.58% 0.00% gdal_translate [kernel.kallsyms] [k] do_page_fault | |
| + 6.58% 1.51% gdal_translate [kernel.kallsyms] [k] __do_page_fault | |
| + 4.93% 0.10% gdal_translate [kernel.kallsyms] [k] handle_mm_fault | |
| + 4.77% 0.33% gdal_translate [kernel.kallsyms] [k] __handle_mm_fault | |
| + 3.75% 0.00% gdal_translate [unknown] [.] 0000000000000000 | |
| + 3.23% 0.06% gdal_translate [kernel.kallsyms] [k] __alloc_pages_nodemask | |
| + 2.65% 0.00% gdal_translate [kernel.kallsyms] [k] entry_SYSCALL_64_after_hwframe | |
| + 2.64% 0.11% gdal_translate [kernel.kallsyms] [k] do_syscall_64 | |
| + 2.13% 0.02% gdal_translate [kernel.kallsyms] [k] __alloc_pages_slowpath | |
| + 2.12% 0.00% gdal_translate libgdal.so.3.1.0 [.] 0x00007f308246fb74 | |
| + 2.11% 0.00% gdal_translate libgdal.so.3.1.0 [.] 0x00007f308246fb63 | |
| + 2.06% 0.00% gdal_translate libgdal.so.3.1.0 [.] 0x00007f308246fb82 | |
| + 1.91% 0.02% gdal_translate [kernel.kallsyms] [k] alloc_pages_vma | |
| + 1.86% 0.01% gdal_translate [kernel.kallsyms] [k] do_try_to_free_pages |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| perf record -g -a -F 100 -e cpu-clock -G docker/${CONTAINER_ID} sleep 90 | |
| perf report |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
A large porition of time is being spent inside of memset inside of alpine, changing the image to ubuntu fixes the problem..
This would imply there is some issue with
muslandgdal