Object Tracking Using Adaptive Diffusion Flow Active Model

— Object tracking is one of the most important topics in the fields of image processing and computer vision. Object tracking is the process of finding interesting moving objects and following them from frame to frame. In this research, Active models–based object tracking algorithm is introduced. Active models are curves placed in an image domain and can evolve to segment the object of interest. Adaptive Diffusion Flow Active Model (ADFAM) is one the most famous types of Active Models. It overcomes the drawbacks of all previous versions of the Active Models specially the leakage problem, noise sensitivity, and long narrow hols or concavities. The ADFAM is well known for its very good capabilities in the segmentation process. In this research, it is adopted for segmentation and tracking purposes. The proposed object tracking algorithm is initiated by detecting the target moving object manually. Then, the ADFAM convergence of the current video frame is reused as an initial estimation for the next video frame and so on. The proposed algorithm is applied to several video sequences, different in terms of the nature of the object, the nature of the background, the speed of the object, object motion direction, and the inter-frame displacement. Experimental results show that the proposed algorithm performed very well and successfully tracked the target object in all different cases.


Introduction
In the latest years, automated controls have affected life in diverse fields. Automation has been applied in space, hospitals, factories, the military, security, etc. The controllers of high performance required big amounts of integrated information about systems and the environment for feedback. This type of data can be provided by the vision system which is a sensor. Vision sensors are used for sensing targets in complex systems such as mobile robots, active and intelligent cruise controls. In a vision system, processing such as object detection and tracking are crucial functions that must be accomplished carefully [1]. Object tracking has been one of the hottest topics in the field of computer vision [2,3]. Object tracking simply means detection (finding) of interesting moving objects and tracking (following) of such objects from frame to frame [4].
The DBT combines detection and tracking and can automatically detect the appearance of new targets or the disappearance of original targets [6][7][8][9][10][11][12]. The drawbacks of this framework are, the need for fixed cameras; and it is difficult to guarantee the stability of these cameras. The need for background modeling-based object detection algorithms; these algorithms face many difficulties such as background initialization (how to get a background image frame without any moving objects) and dynamically changing background [2].
The DFT needs to manually initialize the target object, so it can only track a specified target object. DFT is not affected by the camera's position and does not need background modeling-based object detection algorithms [2,13,14]. In this research, the DFT framework is adopted where, in addition to its advantages, it meets with the research problem, which is tracking a single object.
In this research, active models are used to segment and track the target object. Active models have been advanced as useful means for segmenting rigid and non-rigid objects. Active models was presented in 1987 [15]. They are curves placed or drawn in an image domain that can expand or contract according to internal and external forces of its energy function: The first term of the integral is the internal force that keeps the curve elasticity and cohesion during evolution, while the second term is the external force (image force) that guides the curve toward an object boundary [15].
Because the external force plays an important role in guiding the active model to approach objects boundaries, many new methods for external force have been approved. Among all the methods, that was proposed by Xu et al. [16], has been one of the most powerful external forces, and many modifications have been successfully applied in some interesting applications [17][18][19][20][21][22]. The external force in [16] and its extensions suffer from various dilemmas such as high computational cost, noise sensitivity, parameter sensitivity, lack to the convergence to narrow and deep concavities, and weak edge leakage. In 2013, Wu et al. introduced a new external force that overcame all previously mentioned dilemmas. This external force is called the Adaptive Diffusion Flow (ADF) [23]. Replacing the traditional external force in Eq. (1) with the ADF external force, a new active model is formed, called Adaptive Diffusion Flow Active Model (ADFAM) [23].
In this research, the ADFAM is considered in segmenting and tracking the target moving object.

Adaptive Diffusion Flow External Force
The latest developed external force in [16,24] increases the capture range and has the ability to capture concavities by diffusing the gradient vectors of the edge map computed from the image. Unfortunately, this diffusion process is isotropic and cannot preserve edges. Also, the force field highly depends on the smoothing parameter of the diffusion process. If the smoothing parameter is too large the edges are over smoothed (weak edges fade). If the smoothing parameter is small then no over smoothed edges but the force field preserves excessive noise [23,[25][26][27]. Another problem is the magnitudes of the vectors in the force field. Near the edges, the vectors have large magnitudes. Away from the edges, the vectors have small magnitudes and may be zero. This prevents the active model to capture the long narrow concavities. The ADF external force overcame these problems by using the Harmonic Hypersurface Functional and the Infinity Laplacian Functional.

Harmonic Hypersurface Functional
The Harmonic Hypersurface Functional controls the vectors diffusion process and makes it adaptive. Where in the homogeneous regions (weak edges) the diffusion process is isotropic and in the heterogeneous regions (strong edges) the diffusion process is along the tangent direction of an edge as to maintain the weak edge well [25].
In fact, The Harmonic hypersurface functional consists of a hypersurface minimal functional and the harmonic maps P(x). Hypersurface minimal functional degenerates the uniformly elliptic equation having strong regularizing characteristics in all directions at positions where the contrast of the intensity is weak. And, in a neighbourhood of an edge introducing a strong gradient, the hypersurface minimal functional is preferable to diffuse in the direction parallel to edge works to preserve the weak edge efficiently and give a smooth force field. To obtain a better adaptive diffusion, a Harmonic maps P(x) is used. P(x) ranges from (1 to 2) therefore the vectors diffusion process of the flow field can be modified adaptively as stated by the image characteristics [23,25].
The harmonic hypersurface functional is generalized as f is the edge map of the given image function I which is defined by I: Ω is its boundary. / is the Gaussian smoothing kernel of standard deviation which could smooth the force field. ∇ is the gradient of the curve , ⊗ is the convolution operation [23]. P(.) is a monotonic decreasing function ranging from 1 to 2, where (| |) = 1 + ! !1 |∇2 ! ⊗-(4)| . When |∇ / ⊗ ( )|→0, P(.)→2, this signifies the functional would act as isotropic diffusion within homogeneous regions. When |∇ / ⊗ ( )|→ ∞, P(.)→1, this signifies the diffusion process will be only in the direction parallel the edge works. With this essence, the harmonic hypersurface functional can preserve weak edges and at the same time give a smooth force field [23,25].

Infinity Laplacian Functional
Infinity Laplacian solves the problem of the long narrow concavities, by making the diffusion process be along a normal direction within the image smoothing region. It makes the vectors downward into the boundary concavity rather than converge from two opposite directions, or disappear far from image concavity edges. Infinity Laplacian functional is defined by [23,25]: A minimization of Eq. (4) is named an minimizing Lipschitz interpolant. Both harmonic hypersurface functional and infinity laplacian functional form a new external force called Adaptive Diffusion Flow (ADF). The diffusion is anisotropic. The ADF is defined by [25]: . PQQQQQQQQQQQQQQQQQQQRQQQQQQQQQQQQQQQQQQQS 89::;<=>88 >=>?@A is a monotonic decreasing function, and Θ = | / ⊗ ∇ | # . To preserve the weak edge and concavity convergence simultaneously, the weighting functions must be chosen properly. and h are as in [15] and m is K is a parameter evaluated by K = 1.4826. EL'|∇ | − (|∇ |)'N While the parameter K of the weighting function is assigned by the user according to the state of the edges. If the edges are strong, K must be large and vice versa [21,22]. Figure (1) is an example of the ADF field.

The Proposed Algorithm
The proposed object tracking algorithm uses the ADFAM to segment and tracks the target moving object, as follows: ─ Input: Video sequence. ─ Output: Tracked target object. ─ Step 1: Start ─ Step 2: Object detection: To initiate this algorithm, an object on the first video frame is detected manually (a curve placed inside, outside, or across the object and this curve is the initial ADFAM or initial estimation), the ADFAM parameters must be set, and then initial estimation evolution is performed to segment the object. ─ Step 3: Object tracking: From the second video frame on forth, for each video frame perform steps a) and b): a) Initial estimation placement: The ADFAM convergence (final curve) segmenting the object from the previous video frame is automatically reused and placed over the current video frame as an initial estimation. b) ADFAM evolution: Starting from the initial estimation, a curve evolution is performed on the current video frame to segment the displaced object. ─ Step 4: Stop

Experimental Results and Discussion
The proposed algorithm was applied to several video sequences, different in terms of the nature of the object (rigid, non-rigid), the nature of the background (still, moving), the speed of the object (slow, fast), object motion direction (fixed direction, different directions), and the inter-frame displacement (small, large). The proposed algorithm depends on several parameters which are (α, β, T, σ1, σ2, ADF Iter.No., and ADFAM Iter.

Rigid Object with Still Background
The still background is the background that includes the target moving object only and does not include other objects (blank or empty background), or includes the target moving object and other non-moving objects. The algorithm was applied to two sunrise video sequences. In the first video sequence, the background around the sun was blank.
The direction of the sunrise motion was fixed (to up). As for the speed of the sun, as is known, it is so slow, so we speeded up the video, but the inter-frame displacement remained small and its value was (1). The algorithm was able to track the sunrise motion successfully and easily, see Figure (2). The parameters values are as shown in Table  (1), where we note that the values of all parameters except (k) of the first video frame are large, to increase the capture range of the ADFAM and the force of the ADF field to obtain correct segmentation result and thus achieve a successful tracking process. As for the values of the parameters of the rest of the video frames, they were much smaller than their values in the first frame, due to the stability of the sun shape and the slow change in its position. Increasing the ADFAM capture range was achieved by giving high values to both (σ1) and the (ADFAM Iter.No.). Increasing the force of the ADF force field was done by giving large values to both (σ2) and (ADF Iter.No.). The values of the parameter (k) were (0.03 to 0.05) because the edges of the sun were rather strong. In the second video sequence, the background includes the sun and other non-moving objects, which are the trees. In this experiment, there are three challenges, the first is that the edges of the sun have been occluded by the edges of the trees, and there is also a loose part of the trees that fall into the sun's body (note the original video frame (10) in Figure (3)). This may cause the segmentation process to fail because the loose part of the trees forms strong edges that any of the previous known active models can be attracted to instead of being attracted to the sun edges. The second is, although the video is speeding up, we need 35 video frames during which the sun recovers its occluded edges and moves away completely from the trees, forcing us to increase the inter-frame displacement and therefore a noticeable change and a big difference in the position of the sun. The third is, when the sun is completely away from the trees, the boundaries of the trees form strong edges that it is difficult for the any of the previous known active models to leave to completely lock on around the edges of the sun. The proposed algorithm easily overcame these three challenges and was able to track the sunrise motion efficiently, see the Figure (3). For the first challenge, it was overcome as follows: In the first video frame (10), the edges of the tree covering the sun edges were preserved and detected by decreasing the capture range of the ADFAM and assign an intermediate value to (k). The loose part of the tree was eliminated by increasing the smoothness of the ADF force field by giving (σ2) a large value. The parameters values of the video frame (10) as shown in Table (2). For the second challenge, as for the inter-frame displacement, it was a large (5), where the video frames were used, which included a very noticeable change in the position of the sun. The proposed algorithm overcame this challenge by increasing the capture range of the ADFAM as well as increasing the force of ADF force field in the video frames (15-30). Note Table (2). For the third challenge, the video frame (35) is the frame in which the edges of the sun are completely separated from the edges of the trees. To encourage the ADFAM to leave the edges of the trees and go around the edges of the sun, it was necessary to increase the capture range of the ADFAM and reduce the force of attraction emitted from the edges of those trees. This is done by assigning a very large value to (σ1), (ADFAM Iter.No.), and a very small value to (σ2) respectively. Parameters values of the video frame (35) as shown in Table (2).

Non-rigid Object with Still Background
The algorithm was applied to a video sequence that includes a non-rigid object, which is a moonfish, with a still blank background. The direction of the moonfish motion was fixed (to up). The speed of the moonfish motion was normal. The inter-frame displacement was small (1). Moonfish is a transparent object, its edges change during movement and are very weak concerning its internal structures, as we can see in Figure (4). Despite the condition of the edges, the proposed algorithm succeeded in tracking the motion of the moonfish. Where to avoid the strong edges of the internal structures, the initial estimation must be outside the moonfish, note the initial estimation in Figure  (4). In this case, we completely avoid increasing the force of the ADF field in all the video frames so that the strong edges of the internal structures of the moonfish do not become a source of attraction for the ADFAM and pull it inside. We need to increase the capture range of the ADFAM in the first video frame (01) only, but not for the rest of the video frames. Note Table (3). The value of (σ1) in the video frame (01) is (2) and the (ADFAM Iter.No.) value is (40) while the values of the (σ1) in the rest of the frames were small ranges from (0.01 to 0.5) and the (ADFAM Iter.No.) values are (5). The values of (k) in all video frames except for the first one were small ranging from (0.01 to 0.03) due to the weak edges of the moonfish.

Non-rigid Object with Moving Background
The moving background is the background that includes the target moving object in addition to other objects in motion as well. The algorithm was applied to three white blood cell video sequences. White blood cells are transparent, amoeboid (irregularshaped). Its speed increases when there is an infection. White blood cells move in two ways, either rolling in the blood vessels in search of the infection position or chasing viruses and bacteria at the position of infection. In the latter way (which was considered in this research) the white blood cells move in different directions, where the direction of white blood cell movement changes according to the movement direction of the virus that is chasing it. White blood cells move within a moving background that includes other objects, such as (plasma, platelets, red blood cells, etc.) [18,28]. Three challenges are facing the tracking of the white blood cell motion. The first is that the additional moving objects that surround the white blood cells cause a severe mess. The second is that the red blood cells contain the hemoglobin pigment, so their intensity is higher than the intensity of white blood cells (colorless), and therefore the edges of the red blood cells are very strong compared to the edges of the white blood cells, so any active model is attracted to the edges of the red blood cells, which causes the leakage problem. The third is that the viruses that are eaten by white blood cell form strong edges that can attract any active model into the white blood cell body, causing the segmentation process to fail.
Despite these three challenges, the proposed algorithm was able to efficiently track the motion of white blood cells in these three video sequences. In the first video sequence, the direction of the white blood cell motion was fixed (to down). The speed was slow because it searches for the bacteria or virus at the position of infection (no bacteria or virus nearby). The inter-frame displacement was small and its value was (1). See Figure (5). The parameters values are as shown in Table (4), where we note that the values of all parameters except (k) of the first video frame are large, to increase the capture range of the ADFAM and the force of the ADF field to obtain correct segmentation result and thus achieve a successful tracking process. The parameters values of the rest of the video frames except (ADFAM Iter.No.), were smaller than their values in the first frame, because of the slow change in the shape of the white blood cell as well as its position. The values of the parameter k) were from (0.03 to 0.05) because of the weak edges of the white blood cells.
In the second video sequence, the target white blood cell has very weak edges. Note the original video frame (95) in Figure (6). The speed was also slow due to the absence of infection (no bacteria or virus nearby), but the direction of motion was not fixed. The inter-frame displacement was a large (4), where the video frames were used, which included a very noticeable change in the shape and the position of the white blood cell. As we note in Figure (6) that the white blood cell significantly turns right in the video frame (107) and then heads up to the left in the video frames (111, 115, and 119). In video frame (107), the shape of the white blood cell changes significantly but its position is not. This forced us to increase the capture range of the ADFAM, as well as the force of the ADF field. Note Table ( 5), the parameter values of the video frame (107) became much greater than in the previous video frames (95, 99, and 103). But in the video frames (111, 115, and 119) there is a gradual change in the shape and the position of the white blood cell, so we needed to increase the capture range of ADFAM only, where the (σ1) value was 1.5 and the (ADFAM Iter.No.) values were (10 to 15). It is worth noting that despite the presence of the very weak edges that we mentioned earlier, the proposed algorithm preserved them along the video sequence. The values of the Parameter k) were from (0.02 to 0.05) because of the very weak edges.
In the third video sequence, the motion of the target white blood cell is fast because there is an infection (bacteria), Note the original video frame (10) in Figure (7). The motion direction is not fixed (left to down). Because of the fast motion of the white blood cell, the inter-frame displacement was small (1). As we note in Figure (7), the target white blood cell has very weak edges compared to the edges of the red blood cells that surrounded it. This condition causes a leakage problem. Where the active model will leak from the weak edges of the white blood cell towards the strong edges of the neighboring red blood cell. Despite the presence of this problem, the proposed algorithm was able to track the motion of white blood cells. In this experiment, in addition to the role of the Harmonic Hypersurface Function in preserving the weak edges, we must assign small values to the parameter (k), reduce the force of attraction emitted from the neighboring red blood cells by reducing the force of ADF field, and increase the ADFAM capture range only in a first video frame (10) to achieve a successful segmentation process and thus achieve a successful tracking process. Note Table (6). As for the rest of the video frames, there was no need to increase the ADFAM capture range because of the low value of the inter-frame displacement.
It is important to note that the position of the initial estimation or curve in the first video frame in each of the above three experiments was not outside the white blood cell, or inside it, but rather it was across it. This is to move the initial curve away from the strong edges of the neighboring red blood cells as well as to avoid the strong edges that are inside the body of the white blood cell-like eaten viruses.

Conclusion
In this research, a highly developed active model was used in an important area of computer vision, which is object tracking. The active model was the Adaptive Diffusion Flow Active Model (ADFAM). The target moving object was selected manually, so the tracking algorithm was not fully automatic. The proposed algorithm tracked the motion of the target object by reusing the ADFAM convergence of the current video frame as an initial estimation for the next video frame and so on. Therefore, the object tracking algorithm is not completely subject to the requirements and conditions of well-known standard object tracking algorithms.
In this research, the following was concluded: • The proposed algorithm succeeded in tracking the motion of the object for the following reasons: 1) The Harmonic Hypersurface function, which gives a smooth force field and at the same time preserves the weak edges.
2) The good abilities of the ADFAM, exclusively the insensitivity to the position of the initial estimation. 3) A good set of parameters. 4) A good analysis of the state of the target moving object in the video sequence.
• The object tracking algorithm includes two influential aspects that must be balanced to achieve a successful tracking process. The first aspect is the capture range of the ADFAM and the second aspect is the force or smoothness of the ADF force field.
Concerning the experiments of this research, either they both increase or one of them increases and the other decreases. • The strength of the edges of the target moving object is affected by the strength of the edges of the neighboring objects, whether these neighboring objects are static or moving. The edges of the target moving object may be strong but they are weak compared to the edges of the objects that surround it. Therefore, in the experiments of this research, the highest value for the parameter (k) was (0.05), while the (k) value for strong edges was greater than (0.1).