Framework is capable of running specified WireCell workflow

[knoepfel]

DUNE US S&C R&D item 073

---

This discussion is a continuation of the WireCell discussion in Framework-R-D/phlex-design#3.

[brettviren]

Really, the title should be "DUNE framework is capable of supporting DUNE SNB readouts". It's not a WCT issue per se.

To summarize some Slack chat:

Problem:

• The 100 second of SNB candidate data will not fit in memory and must be processed in chunks.
• Sim and SP must apply a convolution and that naturally increases the time duration of a chunk.
• The "overlap" this caused must be properly handled to avoid introducing artifacts at chunk boundaries.
• The overlap-add/overlap-save (variants of same basic alg) are the standard ways to solve this and conventionally require sequential processing of chunks.
• Neither sequential nor ordered delivery of data is guaranteed in Phlex.

A phlex solution:

1. Perform sim or SP independently on each chunk.
2. Split the "main" and "overlap" parts from each chunk, producing to two adjacent output data cells.
3. Bring "overlap" part from chunk N and "main" part from chunk N+1 together as pairwise "adjacent cells" and do the add.

A benefit of this solution is that WCT sim and SP do not need much change but the overall job becomes a bit more complicated with more WCT/Phlex boundaries:

1. drift + sim but stop at voltage (no digitizer)
2. do the 1->2-> overlap-add
3. apply digitizer
4. apply SP
5. do the 1->2->1 overlap-add

There is a WCT<-->Phlex boundary at 1->2, 2->3 and 4->5. 3+4 can stay in one WCT while real detector data sees only 4 and 5.