2.0 Data-Flow Network Description

Contents:

What is a Data-Flow Network ?
Sub-Network Concept
Iterator Concept
The Pull Technology

What is a Data-Flow Network ?


    A data flow network is the name given to multiple nodes connected together forming a network that understands a certain protocol. This protocol includes initialization, processing and connection methods. A network simply a collection of nodes. It must have an input node and an output node. To get the output from the network, we must ask its output node to get the output of the nodes its connected to, and these output nodes ask to their connected inputs to output the result and so on. So, in other terms, we are pulling the output from the node that will pull its its inputs' outputs, etc. The "getOutput" requests propagates until they reache the first node of a path and then each node of the stacked getOutput requests computes their outputs.  It seems a little bit complicated, but it works the way a resursive function works by stacking the requests. See the graphical representation of the pull technology.

    The goal of the network is to accomplish a certain task by connecting small specific nodes that do a part of the processing together to build a complete processing sequence. That way, any node can be interchanged and modified easily and new algorithms can be created (connecting nodes in a different way) without compiling and building a new executable. All we need is pre-defined nodes with known inputs and outputs and a syntax for connecting them together that a parser can understand.

    In addition to the processing nodes themselves, we added condition nodes and decision nodes that can enable and disable processing paths depending if a certain condition is met of not. That way, we can build much more intelligent networks that only process useful things according to decision rules. 

    To synchronize all those nodes, we must use a sequence number (or a processing number) to make sure the node returns the required output. This number is useful because a node could cache its outputs and outputs of the past (a sequence number lower than the current) can be asked. Instead of processing the request again it only has to return the cached output associated with this sequence number. By default, each node has a cache of size 1. It means it caches the result of the current sequence. If the same sequence number is asked more than one time, the node only has to return (for the Nth time != 1) the cached output without reprocessing. 
 

Sub-Network Concept

The sub-network concept is a powerful way to divide big networks into smaller ones. If we need the same sequence of nodes many times,  you can just define it one time and reuse them. The way to do that is to define the network one time, and to reuse it into a bigger network as a node called a super node. Of course, the sub network must satisfy few requirements :  In that case, the super node inherits its inputs from the input node and its output from the output node. If you look at the actual code of the network, the Network class inherits from Node which means that a network can be treated like a node without any problem if it satisfies the above rules.

Iterator Concept

 
  • An Iterator is in fact a sub network that executes in a loop until a condition is met.
  • We reset the nodes of the Iterator at the start of each new iteration.
  • Depending on the type of the Iterator sub network, the condition can be tested either at the begining or at the end of each iteration.  Ex: WHILE(condition) DO{...} and DO {...}WHILE(condition).
  • To specify the type (condition testing at beginin, or at the end of each iteration) of the Iterator, a subnet parameter is given to the Iterator subnet.  Here is the sytax: 
    • <param: DOWHILE, true> ( in super network)
    • <param: DOWHILE, subnet_param: DOWHILE> (in Iterator sub network)
  • By default, an Iterator subnet is of the type WHILE(condition) DO {...}

 

The Pull Technology

 

 

This is a simple example of of the Pull technology we are using.
 
 

The Pull Network Demonstration
Network: SAMPLE_NET
{

/* We are assuming that NODE1 and NODE2 have an input named INPUT and an output named OUTPUT */

<node: NODE1> <type: NODE1_TYPE>

<node: NODE2> <type: NODE2_TYPE>
<input: INPUT, NODE1, OUTPUT>

}
  • This is a simple exemple that illustrates the calling sequence of the getOutput(...) method. As you can see, we need to have an external agent that asks for the output of Node 2. Since Node 2 needs the output of Node 1 to compute, it asks the output of Node 1 through getOutput(...).
  • In a case where we have more than 2 nodes, each node asks for its input nodes to compute the result before processing.