I talk over a bug from a computer vision prototype from 2009 that required hardware fixes, software fixes, BIOS fixes, Windows Registry fixes, and a tour through distributed queuing.
The recommendation for linear vs parallel pipelines make sense when paired with the decision to append calculations from each node to its input and pass the whole thing along.
As soon as you wrote about passing the input of a node to the output I guessed the cause. Each node should extract data or get out of the way. Otherwise you have a graph of nodes logging verbosely and the next node reading from those logs. Perhaps that is an easier explanation for why the decision to pass along inputs to outputs was a really poor design choice.
Definitely, and good catch on the hint! I wish I saw it as fast as you did haha. I cut this from the post, but the OG use case was offline processing of video collected from airplanes. So it made complete sense to make linear pipelines because they were concerned with throughput and not latency. But for the soft-real-time video system, we cared the most about latency.
The recommendation for linear vs parallel pipelines make sense when paired with the decision to append calculations from each node to its input and pass the whole thing along.
As soon as you wrote about passing the input of a node to the output I guessed the cause. Each node should extract data or get out of the way. Otherwise you have a graph of nodes logging verbosely and the next node reading from those logs. Perhaps that is an easier explanation for why the decision to pass along inputs to outputs was a really poor design choice.
Congratulations on figuring it out!
Definitely, and good catch on the hint! I wish I saw it as fast as you did haha. I cut this from the post, but the OG use case was offline processing of video collected from airplanes. So it made complete sense to make linear pipelines because they were concerned with throughput and not latency. But for the soft-real-time video system, we cared the most about latency.