Author: Chris Lattner
The libdispatch is one of the most misused API due to the way it was presented to us when it was introduced and for many years after that, and due to the confusing documentation and API. This page is a compilation of important things to know if you're going to use this library. Many references are available at the end of this document pointing to comments from Apple's very own libdispatch maintainer (Pierre Habouzit).
My take-aways are:
-
You should create very few, long-lived, well-defined queues. These queues should be seen as execution contexts in your program (gui, background work, ...) that benefit from executing in parallel. An important thing to note is that if these queues are all active at once, you will get as many threads running. In most apps, you probably do not need to create more than 3 or 4 queues.
-
Go serial first, and as you find performance bottle necks, measure why, and if concurrency helps, apply with care, always validating under system pressure. Reuse
Foundation offers a Thread class, internally based on pthread, that can be used to create new threads and execute closures.
// Detaches a new thread and uses the specified selector as the thread entry point.
Thread.detachNewThreadSelector(selector: Selector>, toTarget: Any, with: Any)
// Subclass
class MyThread: Thread {| Requirement already satisfied: wheel in ./.venv/lib/python3.7/site-packages (0.33.6) | |
| Obtaining file:///home/ubuntu/chia-blockchain | |
| Collecting asyncssh | |
| Using cached https://files.pythonhosted.org/packages/cf/3b/9ac19dc6c722f79b97b7878a4ffc872e36e626071e640532a8947649bbb1/asyncssh-2.1.0-py3-none-any.whl | |
| Collecting autoflake | |
| Using cached https://files.pythonhosted.org/packages/88/88/9a8ae06cc31f68b26078cbbef50ae5df361f4ac0efaafbf03a9fa79ca9c5/autoflake-1.3.1.tar.gz | |
| Collecting black | |
| Using cached https://files.pythonhosted.org/packages/fd/bb/ad34bbc93d1bea3de086d7c59e528d4a503ac8fe318bd1fa48605584c3d2/black-19.10b0-py36-none-any.whl | |
| Collecting blspy | |
| Using cached https://files.pythonhosted.org/packages/da/a6/dd8e0395fc25a18289055b843e79eafd9638d64bbd15489f76bf9756222c/blspy-0.1.13.tar.gz |
| Requirement already satisfied: wheel in ./.venv/lib/python3.7/site-packages (0.33.6) | |
| Obtaining file:///home/ubuntu/chia-blockchain | |
| Collecting asyncssh (from chiablockchain==1.2.dev5+g5492bbf.d20191221) | |
| Using cached https://files.pythonhosted.org/packages/cf/3b/9ac19dc6c722f79b97b7878a4ffc872e36e626071e640532a8947649bbb1/asyncssh-2.1.0-py3-none-any.whl | |
| Collecting autoflake (from chiablockchain==1.2.dev5+g5492bbf.d20191221) | |
| Using cached https://files.pythonhosted.org/packages/88/88/9a8ae06cc31f68b26078cbbef50ae5df361f4ac0efaafbf03a9fa79ca9c5/autoflake-1.3.1.tar.gz | |
| Collecting black (from chiablockchain==1.2.dev5+g5492bbf.d20191221) | |
| Using cached https://files.pythonhosted.org/packages/fd/bb/ad34bbc93d1bea3de086d7c59e528d4a503ac8fe318bd1fa48605584c3d2/black-19.10b0-py36-none-any.whl | |
| Collecting blspy (from chiablockchain==1.2.dev5+g5492bbf.d20191221) | |
| Using cached https://files.pythonhosted.org/packages/da/a6/dd8e0395fc25a18289055b843e79eafd9638d64bbd15489f76bf9756222c/blspy-0.1.13.tar.gz |
| (.venv) ➜ chia-blockchain git:(master) ✗ python -m scripts.create_plots -k 30 -n 7 | |
| Creating 7 plots of size 30, sk_seed dd999d5920aceabd81f489642386e09fccb53187f263557f7518e3cf803aa492 ppk <PublicKey 1030db04c277980405f8bb6f34b2381c4c2817e4ee201f62136323a2c66487f3f7d4103ee991a7c5444f1f416c100353> | |
| Plot plot-0-30-e048926f080fbbf06c7f26b40733d7a7ec01a132b39dcec3a093c46850124dc0.dat already exists | |
| Plot plot-1-30-1c24294a8eb4fe58efd9c23f13969fc5eeef4d9357e33f79757ac949e7b76d61.dat already exists | |
| Plot plot-2-30-96d203865c1531056e1155b6bd3a9daece879b9c6b299b1aeedded8c5f858a44.dat already exists | |
| Plot plot-3-30-71a56b50f796b42c6a818f10ba613d13a4af80239e6d67c120e5db8098153628.dat already exists | |
| Plot plot-4-30-8ca4733c3e5bac6dce23572dd30ae1ef7a5584004a740dddda5b3ffc8c5cb885.dat already exists | |
| Plot plot-5-30-3bc4ab9849dffa2411e63507d0e9308babe182d6f2c01b1d44c70037c2694c1b.dat already exists | |
| Starting plotting progress into file /Users/ruslanskorb/chia-blockchain/plots/plot-6-30-42c67a3b864d7526b924602ab1832e04fe82300653 |
| Last login: Sun Dec 8 10:17:16 on console | |
| cd% ➜ ~ cd /Volumes/Chia/chia-blockchain | |
| ➜ chia-blockchain git:(master) ✗ mongod --fork --dbpath ./db/ --logpath mongod.log | |
| about to fork child process, waiting until server is ready for connections. | |
| forked process: 766 | |
| child process started successfully, parent exiting | |
| ➜ chia-blockchain git:(master) ✗ . .venv/bin/activate | |
| (.venv) ➜ chia-blockchain git:(master) ✗ sh ./scripts/run_farming.sh | |
| kill: 810: No such process | |
| Traceback (most recent call last): |
| (.venv) ➜ chia-blockchain git:(master) ✗ python -m scripts.create_plots -k 30 -n 3 | |
| Creating 3 plots of size 30, sk_seed dd999d5920aceabd81f489642386e09fccb53187f263557f7518e3cf803aa492 ppk <PublicKey 1030db04c277980405f8bb6f34b2381c4c2817e4ee201f62136323a2c66487f3f7d4103ee991a7c5444f1f416c100353> | |
| Plot plot-0-30-e048926f080fbbf06c7f26b40733d7a7ec01a132b39dcec3a093c46850124dc0.dat already exists | |
| Starting plotting progress into file /Volumes/Chia/chia-blockchain/plots/plot-1-30-1c24294a8eb4fe58efd9c23f13969fc5eeef4d9357e33f79757ac949e7b76d61.dat. | |
| Memo: | |
| ID: 1c24294a8eb4fe58efd9c23f13969fc5eeef4d9357e33f79757ac949e7b76d61 | |
| Plot size is: 30 | |
| Starting phase 1/4: Forward Propagation... |
| Last login: Thu Dec 5 18:17:32 on ttys001 | |
| ➜ chia-blockchain git:(master) ✗ . .venv/bin/activate | |
| (.venv) ➜ chia-blockchain git:(master) ✗ python -m scripts.create_plots -k 30 -n 1 | |
| Creating 1 plots of size 30, sk_seed dd999d5920aceabd81f489642386e09fccb53187f263557f7518e3cf803aa492 ppk <PublicKey 1030db04c277980405f8bb6f34b2381c4c2817e4ee201f62136323a2c66487f3f7d4103ee991a7c5444f1f416c100353> | |
| Starting plotting progress into file /Users/ruslanskorb/chia-blockchain/plots/plot-0-30-e048926f080fbbf06c7f26b40733d7a7ec01a132b39dcec3a093c46850124dc0.dat. | |
| Memo: | |
| ID: e048926f080fbbf06c7f26b40733d7a7ec01a132b39dcec3a093c46850124dc0 | |
| Plot size is: 30 |
| FullNode src.full_node : INFO 04:14:29.998 This is the first block at height 2106, so propagate. | |
| FullNode src.full_node : INFO 04:14:30.011 New best unfinished block at height 2106 | |
| FullNode src.server.server : INFO 04:14:30.018 -> unfinished_block to peer ('206.80.236.251', 8444) | |
| FullNode src.server.server : INFO 04:14:30.031 -> unfinished_block to peer ('206.80.236.251', 8444) | |
| FullNode src.server.server : INFO 04:14:31.068 <- unfinished_block from peer ('206.80.236.251', 8444) | |
| FullNode src.server.server : INFO 04:14:31.342 <- unfinished_block from peer ('206.80.236.251', 8444) | |
| FullNode src.server.server : INFO 04:14:33.904 <- unfinished_block from peer ('18.179.4.125', 8444) | |
| FullNode src.server.server : INFO 04:14:37.582 <- unfinished_block from peer ('206.80.236.251', 8444) | |
| FullNode src.server.server : INFO 04:14:40.786 Connection with ('18.232.223.201', 8444) established | |
| FullNode src.server.server : INFO 04:14 |