FPGA-SoC Based Object Tracking Algorithms: A Literature Review

Section: Review Paper
Issue
Vol. 28 No. 2 (2023): Volume 28 Issue 2
Published
Sep 1, 2023
Pages
284-295

Abstract

Systems for object detection and tracking are becoming increasingly important in practical applications today. Many research and development groups are interested in improving the performance of such systems, and numerous methods have been developed and proposed. Additionally, computer vision is constantly developing and implemented on reconfigurable and embedded systems. The purpose of this study is to present past and recent research works in the field of visual tracking systems that used FPGA and FPGA-SoC platforms. The study includes a brief description of several popular algorithms related to the main characteristics and in which field is preferred. Resource utilization was also considered in this study to present the most and the least resources used to implement different algorithms. The study found that flip-flops (FF) and lookup tables (LUT) are usually used, while BRAM, DSP, and multipliers had the lowest percentage utilization. Due to the recent development in the production of advanced processing systems, there is an increase focusing on employing FPGA-SoC platforms in visual surveillance systems. The reason behind that is their ability to implement complex processing using both hardware and software co-design to gain high performance in less design time compared with using only FPGA-based platforms.

References

  1. [Horn, B.K. and Schunck, Determining optical flow, 1981, pp. 185203.
  2. A. Arif et al., Performance and energy-efficient implementation of a smart city application on FPGAs, J Real-Time Image Proc, vol. 17, no. 3, pp. 729743, Jun. 2020, doi: 10.1007/s11554-018-0792-x.
  3. A. Yilmaz, O. Javed, and M. Shah, Object tracking: A survey, ACM Comput. Surv., vol. 38, no. 4, p. 13, Dec. 2006, doi: 10.1145/1177352.1177355.
  4. Artur Zawadzki and Marek Gorgon, Automatically controlled pantilt smart camera with FPGA based image analysis system dedicated to real-time tracking of a moving object, Journal of Systems Architecture, 2015.
  5. B. M. K. Younis, B. Sh. Mahmood, and F. H. Ali, Reconfigurable Self-Organizing Neural Network Design and its FPGA Implementation, (AREJ), vol. 17, no. 3, pp. 99115, Jun. 2009, doi: 10.33899/rengj.2009.42925.
  6. C. Stauffer and W. E. L. Grimson, Adaptive background mixture models for real-time tracking, in Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins, CO, USA: IEEE Comput. Soc, 1999, pp. 246252. doi: 10.1109/CVPR.1999.784637.
  7. C. Wang, E. D. Burnham-Fay, and J. D. Ellis, Real-time FPGA-based Kalman filter for constant and non-constant velocity periodic error correction, Precision Engineering, vol. 48, pp. 133143, Apr. 2017, doi: 10.1016/j.precisioneng.2016.11.013.
  8. C. Zhang, P. Li, G. Sun, Y. Guan, B. Xiao, and J. Cong, Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks, in Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey California USA: ACM, Feb. 2015, pp. 161170. doi: 10.1145/2684746.2689060.
  9. D. B. K. Trieu and T. Maruyama, Real-time color image segmentation based on mean shift algorithm using an FPGA, J Real-Time Image Proc, vol. 10, no. 2, pp. 345356, Jun. 2015, doi: 10.1007/s11554-012-0319-9.
  10. Dr. F. Ali, Transformation Matrix for 3D computer Graphics Based on FPGA(English), (AREJ), vol. 20, no. 5, pp. 115, Oct. 2012, doi: 10.33899/rengj.2012.61024.
  11. Dr. S. A. Dawwd and U. T. Salim, Systolic Video Stream Object Detector Using FPGA-E, (AREJ), vol. 22, no. 4, pp. 3343, Sep. 2014, doi: 10.33899/rengj.2014.89977.
  12. F. Barranco, M. Tomasi, J. Diaz, M. Vanegas, and E. Ros, Parallel Architecture for Hierarchical Optical Flow Estimation Based on FPGA, IEEE Trans. VLSI Syst., vol. 20, no. 6, pp. 10581067, Jun. 2012, doi: 10.1109/TVLSI.2011.2145423.
  13. F. Kristensen, H. Hedberg, H. Jiang, P. Nilsson, and V. wall, An Embedded Real-Time Surveillance System: Implementation and Evaluation, J Sign Process Syst Sign Image Video Technol, vol. 52, no. 1, pp. 7594, Jul. 2008, doi: 10.1007/s11265-007-0100-7.
  14. G. Conti, M. Quintana, P. Malagn, and D. Jimnez, An FPGA Based Tracking Implementation for Parkinsons Patients, Sensors, vol. 20, no. 11, p. 3189, Jun. 2020, doi: 10.3390/s20113189.
  15. G. K. Gultekin and A. Saranli, An FPGA based high performance optical flow hardware design for computer vision applications, Microprocessors and Microsystems, vol. 37, no. 3, pp. 270286, May 2013, doi: 10.1016/j.micpro.2013.01.001.
  16. H.-S. Seong, C. E. Rhee, and H.-J. Lee, A Novel Hardware Architecture of the LucasKanade Optical Flow for Reduced Frame Memory Access, IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 6, pp. 11871199, Jun. 2016, doi: 10.1109/TCSVT.2015.2437077.
  17. HARRIS, C and STEPHENS, A combined corner and edge detector, presented at the Vision Conference, 1988, pp. 147151.
  18. Hongtu Jiang, H. Ardo, and V. Owall, A Hardware Architecture for Real-Time Video Segmentation Utilizing Memory Reduction Techniques, IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 2, pp. 226236, Feb. 2009, doi: 10.1109/TCSVT.2008.2009244.
  19. I. Bravo, M. Mazo, J. L. Lzaro, A. Gardel, P. Jimnez, and D. Pizarro, An Intelligent Architecture Based on Field Programmable Gate Arrays Designed to Detect Moving Objects by Using Principal Component Analysis, Sensors, vol. 10, no. 10, pp. 92329251, Oct. 2010, doi: 10.3390/s101009232.
  20. I. Ishii, T. Taniguchi, K. Yamamoto, and T. Takaki, High-Frame-Rate Optical Flow System, IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 1, pp. 105112, Jan. 2012, doi: 10.1109/TCSVT.2011.2158340.
  21. J. G. Pandey, An embedded FPGA-SoC framework and its usage in moving object tracking application, Des Autom Embed Syst, vol. 25, no. 3, pp. 213236, Sep. 2021, doi: 10.1007/s10617-021-09252-y.
  22. K. Blachut and T. Kryjak, Real-Time Efficient FPGA Implementation of the Multi-Scale Lucas-Kanade and Horn-Schunck Optical Flow Algorithms for a 4K Video Stream, Sensors, vol. 22, no. 13, p. 5017, Jul. 2022, doi: 10.3390/s22135017.
  23. L. Bai, Y. Zhao, and X. Huang, A CNN Accelerator on FPGA Using Depthwise Separable Convolution, IEEE Trans. Circuits Syst. II, vol. 65, no. 10, pp. 14151419, Oct. 2018, doi: 10.1109/TCSII.2018.2865896.
  24. M. Amiri, F. M. Siddiqui, C. Kelly, R. Woods, K. Rafferty, and B. Bardak, FPGA-Based Soft-Core Processors for Image Processing Applications, J Sign Process Syst, vol. 87, no. 1, pp. 139156, Apr. 2017, doi: 10.1007/s11265-016-1185-7.
  25. M. Genovese and E. Napoli, FPGA-based architecture for real time segmentation and denoising of HD video, J Real-Time Image Proc, vol. 8, no. 4, pp. 389401, Dec. 2013, doi: 10.1007/s11554-011-0238-1.
  26. M. Genovese, E. Napoli, D. De Caro, N. Petra, and A. G. M. Strollo, FPGA Implementation of Gaussian Mixture Model Algorithm for 47 fps Segmentation of 1080p Video, Journal of Electrical and Computer Engineering, vol. 2013, pp. 18, 2013, doi: 10.1155/2013/129589.
  27. M. Tomasi, S. Pundlik, and G. Luo, FPGADSP co-processing for feature tracking in smart video sensors, J Real-Time Image Proc, vol. 11, no. 4, pp. 751767, Apr. 2016, doi: 10.1007/s11554-014-0413-2.
  28. N. A. Mandellos, I. Keramitsoglou, and C. T. Kiranoudis, A background subtraction algorithm for detecting and tracking vehicles, Expert Systems with Applications, vol. 38, no. 3, pp. 16191631, Mar. 2011, doi: 10.1016/j.eswa.2010.07.083.
  29. P. Babu and E. Parthasarathy, FPGA implementation of multi-dimensional Kalman filter for object tracking and motion detection, Engineering Science and Technology, an International Journal, vol. 33, p. 101084, Sep. 2022, doi: 10.1016/j.jestch.2021.101084.
  30. P. Hobden, S. Srivastava, and E. Nurellari, FPGA-Based CNN for Real-Time UAV Tracking and Detection, Front. Space Technol., vol. 3, p. 878010, May 2022, doi: 10.3389/frspt.2022.878010.
  31. R. Marzotto, P. Zoratti, D. Bagni, A. Colombari, and V. Murino, A real-time versatile roadway path extraction and tracking on an FPGA platform, Computer Vision and Image Understanding, vol. 114, no. 11, pp. 11641179, Nov. 2010, doi: 10.1016/j.cviu.2010.03.015.
  32. R. Parekh, Fundamentals of image, audio, and video processing using MATLAB: with applications to pattern recognition, First edition. Boca Raton London New York: CRC Press, 2021.
  33. R. Rodriguez-Gomez, E. J. Fernandez-Sanchez, J. Diaz, and E. Ros, FPGA Implementation for Real-Time Background Subtraction Based on Horprasert Model, Sensors, vol. 12, no. 1, pp. 585611, Jan. 2012, doi: 10.3390/s120100585.
  34. S. Gong, C. Liu, Y. Ji, B. Zhong, Y. Li, and H. Dong, Advanced Image and Video Processing Using MATLAB, vol. 12. in Modeling and Optimization in Science and Technologies, vol. 12. Cham: Springer International Publishing, 2019. doi: 10.1007/978-3-319-77223-3.
  35. S. Guo et al., A system on chip-based real-time tracking system for amphibious spherical robots, International Journal of Advanced Robotic Systems, vol. 14, no. 4, p. 172988141771655, Jul. 2017, doi: 10.1177/1729881417716559.
  36. S. Sajjanar, S. K. Mankani, P. R. Dongrekar, N. S. Kumar, Mohana, and H. V. Ravish Aradhya, Implementation of real time moving object detection and tracking on FPGA for video surveillance applications, in 2016 IEEE Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER), Mangalore: IEEE, Aug. 2016, pp. 289295. doi: 10.1109/DISCOVER.2016.7806248.
  37. Sirpa Korhonen, Hardware Accelerated Visual Tracking Algorithms. A Systematic Literature Review, Aug. 2015. [Online]. Available: https://www.researchgate.net/publication/281088238
  38. U. Ali and M. B. Malik, Hardware/software co- design of a real-time kernel based tracking system, Journal of Systems Architecture, vol. 56, no. 8, pp. 317326, Aug. 2010, doi: 10.1016/j.sysarc.2010.04.008.
  39. Z. Wei, D.-J. Lee, and B. E. Nelson, FPGA-based Real-time Optical Flow Algorithm Design and Implementation, JMM, vol. 2, no. 5, pp. 3845, Sep. 2007, doi: 10.4304/jmm.2.5.38-45.

Authors

Marwan Abdulkhaleq Al-yoonus

Electrical Engineering Departement, College of Engineering, University of Mosul, Mosul, Iraq

ORCID
Saad Ahmed Al-Kazzaz

Department of Mechatronics Engineering, College of Engineering, University of Mosul, Mosul, Iraq

ORCID

Identifiers

https://doi.org/10.33899/rengj.2023.138936.1243
Download this PDF file
PDF

Statistics

How to Cite

[1]
M. Abdulkhaleq Al-yoonus and S. Ahmed Al-Kazzaz, “FPGA-SoC Based Object Tracking Algorithms: A Literature Review”, AREJ, vol. 28, no. 2, pp. 284–295, Sep. 2023.
Crossref
Scopus
Google Scholar
Europe PMC