diff --git a/docs/module-01-setup.md b/docs/module-01-setup.md index 89c7993..78563b1 100644 --- a/docs/module-01-setup.md +++ b/docs/module-01-setup.md @@ -53,6 +53,7 @@ Environments made by pixi are essentially the same as the ones done with conda. === "Option A: conda" ```bash + unset PYTHONPATH conda env create -f environment.yml conda activate vine-cms-analysis-stack ``` diff --git a/docs/module-04-scale-out.md b/docs/module-04-scale-out.md index 6fb004a..0556492 100644 --- a/docs/module-04-scale-out.md +++ b/docs/module-04-scale-out.md @@ -195,21 +195,23 @@ them. ```bash conda activate vine-cms-analysis-stack - vine_factory -T condor -C factory.json -E '--single-shot' \ + vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ``` === "Option B: pixi" ```bash - pixi run vine_factory -T condor -C factory.json -E '--single-shot' \ + pixi run vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ``` - `-T condor`: submit workers as HTCondor jobs. - `-C factory.json`: the configuration above. -- `-E '--single-shot'`: passed on to every `vine_worker`, as in Module 3, - so that the worker exits as soon as its manager goes away. +- `-E '--single-shot --connection-mode by_apparent_ip'`: passed on to every + `vine_worker`. `--single-shot`, as in Module 3, makes the worker exit as + soon as its manager goes away; `--connection-mode by_apparent_ip` makes it + connect to the manager's address as seen by the catalog. - `--wrapper ./with_oasis_certs --wrapper-input with_oasis_certs`: send the wrapper script along with each job, and start the worker through it (see [Certificates](#certificates) below). @@ -227,7 +229,7 @@ cores: 4.000 factory-timeout: 120 MB memory: 8000 MB disk: 16000 MB -worker-extra-options: --single-shot +worker-extra-options: --single-shot --connection-mode by_apparent_ip 2026/09/14 11:09:42: |submitted: 0 |needed: 2 |waiting connection: 0 |requested: 2 PROJECT HOST PORT WAITING RUNNING COMPLETE WORKERS -trijet-tuto condorfe.crc.nd.edu 45697 5 0 0 0 diff --git a/docs/module-07-result-postprocess.md b/docs/module-07-result-postprocess.md index f9c200f..264168f 100644 --- a/docs/module-07-result-postprocess.md +++ b/docs/module-07-result-postprocess.md @@ -100,7 +100,7 @@ def accumulate_skims(a, b): before that result would be sent back: ```python -SKIM_DIR = f"/project01/ndcms/users/{getpass.getuser()}/trijet-skims" +SKIM_DIR = f"/project01/ndcms/{getpass.getuser()}/trijet-skims" def make_result_postprocess(skim_dir): def result_postprocess(skim): @@ -258,14 +258,14 @@ mounts `/project01`, so Module 4's `factory.json` works as it is. ```bash conda activate vine-cms-analysis-stack - vine_factory -T condor -C factory.json -E '--single-shot' \ + vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ``` === "Option B: pixi" ```bash - pixi run vine_factory -T condor -C factory.json -E '--single-shot' \ + pixi run vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ``` @@ -277,14 +277,14 @@ mounts `/project01`, so Module 4's `factory.json` works as it is. ```bash conda activate vine-cms-analysis-stack - CPATH=/usr/include vine_factory -T condor -C factory.json -E '--single-shot' \ + CPATH=/usr/include vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ``` === "Option B: pixi" ```bash - CPATH=/usr/include pixi run vine_factory -T condor -C factory.json -E '--single-shot' \ + CPATH=/usr/include pixi run vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ``` @@ -303,7 +303,7 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all.
TaskVine manager 'btovar-trijet-tutorial' listening on port 43775 - skims will be written under /project01/ndcms/users/btovar/trijet-skims + skims will be written under /project01/ndcms/btovar/trijet-skims waiting for workers from your vine_factory... ... success processor skim/SingleMu_Run2012B_195842-196531 ...2-196531/file_2.root[50000:99449] (id=c37dc2d6) cores=1(1) memory_mb=315(2016) wall_time_s=1(0) @@ -314,17 +314,17 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all. reductions [ ██████████████████████████████ ] 11/0/11 events [ ██████████████████████████████ ] 3736416(3736416)/0/3736416 datasets [ ██████████████████████████████ ] 5/5 - 51491 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.d293f06c26b34a6881ef56cacb6479f4.parquet - 78200 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.628e1ba07e4444edb3c4d91f3e73b974.parquet - 104495 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.22bad5ac4d604a428acfe8cecb3fadca.parquet - 101615 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.f29ed1a0abe944db967f5aaa56039250.parquet - 20949 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.3baf79829f824684a95aed90c7f4bfff.parquet - 21085 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.3f7e0817258a44b49ca1384db2c09200.parquet - 105864 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.1a9f144626af4f4eb419cc4e42e77ad1.parquet - 85146 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.bcf1498ba042400da20e05c8bcf0ef3e.parquet - 29110 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.3dbcdfeebd3242a2b45106904efff75f.parquet - 71729 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.e1e6b2219d0d4b3cba1fc6847e1a513d.parquet - 106502 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.885b37f41b764cbbb7a3af8bf3e3821d.parquet + 51491 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.d293f06c26b34a6881ef56cacb6479f4.parquet + 78200 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.628e1ba07e4444edb3c4d91f3e73b974.parquet + 104495 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.22bad5ac4d604a428acfe8cecb3fadca.parquet + 101615 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.f29ed1a0abe944db967f5aaa56039250.parquet + 20949 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.3baf79829f824684a95aed90c7f4bfff.parquet + 21085 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.3f7e0817258a44b49ca1384db2c09200.parquet + 105864 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.1a9f144626af4f4eb419cc4e42e77ad1.parquet + 85146 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.bcf1498ba042400da20e05c8bcf0ef3e.parquet + 29110 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.3dbcdfeebd3242a2b45106904efff75f.parquet + 71729 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.e1e6b2219d0d4b3cba1fc6847e1a513d.parquet + 106502 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.885b37f41b764cbbb7a3af8bf3e3821d.parquet 11 final results, 776186 events in the skim
@@ -344,7 +344,7 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all.
TaskVine manager 'btovar-trijet-tutorial' listening on port 43775 - skims will be written under /project01/ndcms/users/btovar/trijet-skims + skims will be written under /project01/ndcms/btovar/trijet-skims waiting for workers from your vine_factory... ... success processor skim/SingleMu_Run2012B_195842-196531 ...2-196531/file_2.root[50000:99449] (id=e90bb2a9) cores=1(1) memory_mb=881(2016) wall_time_s=7(0) @@ -355,17 +355,17 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all. reductions [ ██████████████████████████████ ] 11/0/11 events [ ██████████████████████████████ ] 3736416(3736416)/0/3736416 datasets [ ██████████████████████████████ ] 5/5 - 51491 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.87dd1353bb14470684ccf368d04a7db0.parquet - 78200 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.d9e238285510447e9bc0658a7508ba5a.parquet - 98205 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.253b21ea0c024132994b9070c7be72b5.parquet - 27239 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.84e438e287ae4dc58c90a247733bc507.parquet - 101616 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.4b1ce447a3cb482e9ecac5ae4c2a1518.parquet - 105864 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.a91bf258aa7b4c3fb7f81cad86594243.parquet - 21086 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.49c3ed3774804c8ba892b840b8e4804f.parquet - 85146 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.d75589d1d3e24242be17082a013c47a2.parquet - 29111 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.a2906dbe00cd4ac88425c4735b3f43f7.parquet - 74400 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.5e8f8fa5d9314a31af54c22c20453151.parquet - 103831 events -> /project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.06d2ffcf22594d159918439b879fc1ba.parquet + 51491 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.87dd1353bb14470684ccf368d04a7db0.parquet + 78200 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_193834-194270.d9e238285510447e9bc0658a7508ba5a.parquet + 98205 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.253b21ea0c024132994b9070c7be72b5.parquet + 27239 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.84e438e287ae4dc58c90a247733bc507.parquet + 101616 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.4b1ce447a3cb482e9ecac5ae4c2a1518.parquet + 105864 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.a91bf258aa7b4c3fb7f81cad86594243.parquet + 21086 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194790-195398.49c3ed3774804c8ba892b840b8e4804f.parquet + 85146 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.d75589d1d3e24242be17082a013c47a2.parquet + 29111 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195399-195841.a2906dbe00cd4ac88425c4735b3f43f7.parquet + 74400 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.5e8f8fa5d9314a31af54c22c20453151.parquet + 103831 events -> /project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_195842-196531.06d2ffcf22594d159918439b879fc1ba.parquet 11 final results, 776189 events in the skim
@@ -411,7 +411,7 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all. "processor_name": "skim", "dataset_name": "SingleMu_Run2012B_194271-194789", "locator": { - "path": "/project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.22bad5ac4d604a428acfe8cecb3fadca.parquet", + "path": "/project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.22bad5ac4d604a428acfe8cecb3fadca.parquet", "events": 104495 } } @@ -447,7 +447,7 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all. "processor_name": "skim", "dataset_name": "SingleMu_Run2012B_194271-194789", "locator": { - "path": "/project01/ndcms/users/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.253b21ea0c024132994b9070c7be72b5.parquet", + "path": "/project01/ndcms/btovar/trijet-skims/SingleMu_Run2012B_194271-194789.253b21ea0c024132994b9070c7be72b5.parquet", "events": 98205 } } @@ -465,7 +465,7 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all. The skim itself is on shared storage. From the login node: ```bash - ls -lgo /project01/ndcms/users/$USER/trijet-skims/ + ls -lgo /project01/ndcms/$USER/trijet-skims/ ``` ``` @@ -493,7 +493,7 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all. The skim itself is on shared storage. From the login node: ```bash - ls -lgo /project01/ndcms/users/$USER/trijet-skims/ + ls -lgo /project01/ndcms/$USER/trijet-skims/ ``` ``` @@ -520,7 +520,7 @@ This time there are 83 chunks of up to 50,000 events, five datasets in all. Every run adds new files (new uuids) next to the old ones. `results/` is wiped at the start of each run, but `/project01/.../trijet-skims/` is not. Clean it up yourself between - attempts: `rm /project01/ndcms/users/$USER/trijet-skims/*.parquet`. + attempts: `rm /project01/ndcms/$USER/trijet-skims/*.parquet`. When you're done, stop the factory with Ctrl-C and check `condor_q`, as in [Module 4](module-04-scale-out.md#clean-up). diff --git a/docs/module-08-multiple-processors.md b/docs/module-08-multiple-processors.md index 0d1ff06..56e7503 100644 --- a/docs/module-08-multiple-processors.md +++ b/docs/module-08-multiple-processors.md @@ -153,14 +153,14 @@ Use the same two terminals as in [Module 4](module-04-scale-out.md). ```bash conda activate vine-cms-analysis-stack - vine_factory -T condor -C factory.json -E '--single-shot' \ + vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ``` === "Option B: pixi" ```bash - pixi run vine_factory -T condor -C factory.json -E '--single-shot' \ + pixi run vine_factory -T condor -C factory.json -E '--single-shot --connection-mode by_apparent_ip' \ --wrapper ./with_oasis_certs --wrapper-input with_oasis_certs ```