@MastersThesis{Andrade:2022:DeCoVe,
author = "Andrade, Rafael Marinho",
title = "Detec{\c{c}}{\~a}o de comportamentos de ve{\'{\i}}culos a
partir de imagens de drones e de monitoramento",
school = "Instituto Nacional de Pesquisas Espaciais (INPE)",
year = "2022",
address = "S{\~a}o Jos{\'e} dos Campos",
month = "2022-05-20",
keywords = "vis{\~a}o computacional, aeronaves n{\~a}o tripuladas, drones,
detec{\c{c}}{\~a}o de comportamentos, intelig{\^e}ncia
artificial, computer vision, unmanned aerial vehicles, drones,
behaviour detection, artificial inteligence.",
abstract = "A {\'a}rea de sensoriamento remoto tem se beneficiado j{\'a}
h{\'a} d{\'e}cadas de imagens obtidas acima do n{\'{\i}}vel do
solo, seja a alguns metros ou milhares quil{\^o}metros de altura,
por aeronaves e sat{\'e}lites, sendo t{\~a}o logo consideradas
essenciais para as aplica{\c{c}}{\~o}es em tal {\'a}rea da
ci{\^e}ncia. As aplica{\c{c}}{\~o}es fazem uso dessas imagens
para extra{\c{c}}{\~a}o de informa{\c{c}}{\~o}es e ent{\~a}o
tomada de decis{\~o}es. Mais recentemente, a crescente
popularidade dos drones fez com que sua aplica{\c{c}}{\~a}o
fosse considerada para praticamente qualquer problema
conceb{\'{\i}}vel, e logo assim passou a ser aplicado para o
sensoriamento remoto em diversas {\'a}reas, inclusive a
cient{\'{\i}}fica. Uma dessas aplica{\c{c}}{\~o}es {\'e} o
monitoramento, que vem sendo aprimorado e automatizado de acordo
com os avan{\c{c}}os de t{\'e}cnicas de intelig{\^e}ncia
computacional, que pouco a pouco passaram a permitir a
identifica{\c{c}}{\~a}o e classifica{\c{c}}{\~a}o de objetos
de interesse, assim como seus rastreios. Apesar disso, a
detec{\c{c}}{\~a}o do comportamento desses objetos ainda {\'e}
uma {\'a}rea relativamente pouco explorada e, considerando a
aplica{\c{c}}{\~a}o em imagens a{\'e}reas obtidas por drones,
as pesquisas s{\~a}o ainda mais escassas. Este trabalho considera
a aplica{\c{c}}{\~a}o de t{\'e}cnicas de vis{\~a}o
computacional para detec{\c{c}}{\~a}o de comportamentos em
imagens a{\'e}reas obtidas por drones, compreendendo a
realiza{\c{c}}{\~a}o de um estudo de caso onde foram capturados
mais de 300000 quadros de v{\'{\i}}deo contendo imagens
rodovi{\'a}rias na regi{\~a}o do Vale do Para{\'{\i}}ba
(S{\~a}o Paulo) e foi desenvolvida uma aplica{\c{c}}{\~a}o para
a detec{\c{c}}{\~a}o de comportamentos de ve{\'{\i}}culos em
rodovias partindo da captura dessas imagens, seguindo para a
detec{\c{c}}{\~a}o e classifica{\c{c}}{\~a}o de
ve{\'{\i}}culos com a rede convolucional profunda YOLOv4, seus
rastreios com o algoritmo Deep SORT, e ent{\~a}o
extra{\c{c}}{\~a}o de perfis comportamentais baseados nas
caracter{\'{\i}}sticas vetoriais de seus deslocamentos, que
s{\~a}o classificados como comportamentos normais ou anormais por
redes de mem{\'o}rias de curto e longo prazo (LSTM) e redes de
m{\'u}ltiplos perceptrons (MLP), processo este explorado por uma
s{\'e}rie de testes e experimentos sob o formato de prova de
conceito. Foram atingidos por fim resultados com 99,76% e 94,58%
de acur{\'a}cia nas tarefas de detec{\c{c}}{\~a}o e
classifica{\c{c}}{\~a}o de ve{\'{\i}}culos, respectivamente,
com 94,53% dos rastreios observados sendo cont{\'{\i}}nuos. Os
m{\'e}todos de discrimina{\c{c}}{\~a}o de comportamentos
abordados apresentaram bons resultados ao serem considerados
cen{\'a}rios est{\'a}ticos, onde {\'e} treinada uma rede para
cada cen{\'a}rio, ainda que com dificuldades de
generaliza{\c{c}}{\~a}o entre cen{\'a}rios distintos, de modo
que as solu{\c{c}}{\~o}es n{\~a}o se provaram robustas e
confi{\'a}veis o suficiente para que uma {\'u}nica rede seja
aplic{\'a}vel em diversos cen{\'a}rios distintos ou em
cen{\'a}rios cuja a perspectiva da captura seja vari{\'a}vel,
como em c{\^a}meras m{\'o}veis e aeronaves em deslocamento.
ABSTRACT: The remote sensing area has been benefiting for decades
from images obtained above ground level, either a few meters or
thousands of kilometers high, by aircraft and satellites, and they
soon got considered essential for applications in such area of
science. Those applications make use of these images to extract
information and then make decisions. More recently, the increasing
popularity of drones meant that its application is considered for
almost any conceivable problem, and soon it started to be applied
for remote sensing in several areas, including the scientific
area. One of these applications is monitoring, which has been
improved and automated according to the advances on computational
intelligence techniques, which little by little started to allow
the identification and classification of objects of interest, as
well as their tracking. Nevertheless, the behaviour detection from
those objects is still a relatively unexplored area and,
considering the application in aerial images obtained by drones,
the researches are even more scarce. This work considers the
application of computer vision techniques to detect behaviours in
aerial images obtained by drones, comprising the realization of a
case study where were captured more than 300000 video frames from
highway footage in the Vale do Para{\'{\i}}ba region (S{\~a}o
Paulo, Brazil) and was developed an application for the detection
of vehicle behaviours on highways, starting from the capture of
the given images, proceeding to the vehicles detection and
classification with the YOLOv4 deep neural convolutional network,
their tracking withe the Deep SORT algorithm and then the
extraction of behavioural profiles based on the vectorial
characteristics of their displacements, which are classified as
normal or abnormal behaviours by long-short term memories (LSTM)
and multilayer perceptrons networks (MLP), a process explored by
several tests and experiments as a proof-of-concept. At last, the
results reached an accuracy of 99.76% and 94.58% at the vehicles
detection and classification tasks, respectively, with 94.53% of
them being continuously tracked. The approached behaviour
discrimantion methods presented good results in static scenarios,
where its trained a network for each scenario, albeit with
generalization issues between distinct scenarios, in a way where
the solutions arent robust and reliable enough to be applied a
single network in several distinct scenarios or scenarios where
the footage perspective is variable, such as from moving cameras
and aircrafts.",
committee = "K{\"o}rting, Thales Sehn (presidente) and Shiguemori, Elcio
Hideiti (orientador) and Santos, Rafael Duarte Coelho dos
(orientador) and Santiago Junior, Valdivino Alexandre and
M{\'a}ximo, Marcos Ricardo Omena de Albuquerque",
englishtitle = "Vehicles behaviour detection from drones and surveillance images",
language = "pt",
pages = "195",
ibi = "8JMKD3MGP3W34T/47AFT2H",
url = "http://urlib.net/ibi/8JMKD3MGP3W34T/47AFT2H",
targetfile = "publicacao.pdf",
urlaccessdate = "23 maio 2024"
}