The modeling of 3D interactive objects from multisensory data The     long-term     goal     of     this     research program   is   to   develop   enhanced   sensing and   modeling   techniques   for   applications that   require   higher   realism   by   means   of   3D object   models   based   on   a   combination   of real    multisensory    measurements,    along with    their    integration,    representation    and interpretation.

Computational intelligence methods

for complex data processing

This   research   program   the   advancement   of computational    intelligence    techniques    and their     integration     with     other     modern     or classical     algorithms     to     create     intelligent solutions     capable     to     ensure     automated processing    and    full    use    of    the    growing volume   of   complex   data   in   a   timely   manner. Collaborations     with     the     IBM     Center     of Analytics   and   Business   Performance   of   the University    of    Ottawa,    and    Ottawa    Police Services; Larus Technologies.

Saliency for modeling  and pattern

recognition in images and in 3D data

Research    project    for    the    development    of hybrid   algorithms   that   combine   human   visual attention    features    and    saliency    detection algorithms   with   modern   image   and   3D   data processing techniques.
Human    motion    quantification    for natural human-machine interfaces The   project   aims   at   the   development   of a   vision   system   to   recuperate   and   track human   gestures   and   postures   for   the natural   control   of   software   applications based   on   gesture   recognition   and   the natural control of robots by imitation. Modeling and use of virtual environments for the psychological treatment of phobias The   research   project   aims   at   adaptively and   automatically   adjusting   the   level-of- detail    of    objects    and    avatars    in    an immersive      environment      for      phobia treatment.     Collaboration    with    prof.    S. Bouchard, UQO Cyberpsychology lab

Research

© Ana-Maria Cretu 2016

Ana-Maria Cretu

                                         ing. jr., Ph.D.
Students wishing to pursue their graduate studies or work on a final project on a subject related to computational intelligence, machine learning, intelligent image processing, biologically-inspired modeling, visual attention models, 3D object modeling, virtual reality, human- computer interaction, intelligent systems development are encouraged to contact directly Dr. Ana-Maria Cretu.

Additional Information:

Programs at UQO  Admission information Financial support:      •  NSERC      •  FRQNT       •  Bourses UQO  T.E.A. Oliveira (Ph.D) G. Rouhafzay (Ph.D) F. Audet (Ph.D) B. Tawbe (Ph.D) J.C. Davila Mesa* (Ph.D) T. Layadi (Ph.D) F. Hui* (M.A.Sc) S. Filiatrault (M.A.Sc) A. Mageau-Pétrin (M.Sc) M. Chagnon-Forget (M.A.Sc) G. Plouffe (Master) N. Pedneault (Master) A. Quenneville (capstone project) T.G. Nkuzimana (internship) J.-M. Hébert (internship)

Alumni:

C. Viau* (M.A.Sc) J. McCausland* (M.A.Sc) M.I. Sina* (M.A.Sc) Capstone projects: P. Roy-Villeneuve F. Gorman N. Duchaine-Ritchot L. Couture-Niles A. Huot M. Chagnon-Forget J.-P. Gauthier L.-P. Fillion G. Plouffe P.-P. Chhay M. Cavrag V. D’Aoust M.-A. Charpentier H. Monette-Thériault S. Filiatrault S. Gauthier J. Tremblay-Gosselin Research Interns: C. C. Diaw N. Bélanger L.H. Franc* T.T. Firmino de Lima (Mitacs Globalink internship) V. Bairaboina (Mitacs Globalink internship) P. Gauthier  Q. Le Délas  I. Filali* T. Level* Y.-F. Nassala* *co-supervision

Research Projects

Sensor systems for multisensory data acquisition

This   project   explores   sensing   technologies   as   well   as   data   integration techniques   for   3D   deformable   object   modeling.   In   this   context,   a   data- driven     neural-network-based     model     for     capturing     implicitly     an d predicting   3D   deformations   of   a   soft   object’s   surface   subject   to   external forces   is   proposed.   Visual   data,   in   form   of   3D   point   clouds   gathered with    a    Kinect    sensor,    is    collected    over    an    object    while    forces    are exerted   by   means   of   the   probing   tip   of   a   force-torque   sensor.   A   novel approach   based   on   neural   gas   fitting   is   then   proposed   to   describe   the particularities   of   a   deformation   over   the   selectively   simplified   3D   surface of   the   object   without   requiring   knowledge   on   the   object   material.   An Elman   neural   network   is   finally   trained   to   predict   the   mapping   between the   measured   parameters   characterizing   the   interaction   with   the   obj ect and the change in the object shape due to the interaction. Support: Related publications: B.   Tawbe    and   A.-M.   Cretu,   “Acquisition   and   Prediction   of   Soft   Object   Shape   Using   a   Kinect   and   a   Force- Torque Sensor”, submitted to IEEE Trans. Instrumentation and Measurement , June 2016, NEW . T he   ill-defined   nature   of   tactile   information   turns   tactile sensors   into   th e   last   frontier   to   robots   that   can   handle everyday    objects    and    interact    with    humans    through contact.   To   overcome   this   frontier   many   design   aspects have       to       be       considered:       sensors       placement, electronic/mechanical   hardware,   methods   to   access   and acquire   signals,   calibration,   and   algorithms   to   process   and interpret   sensing   data   in   real   time. This   project   aims   at   the design    and    hardware    implementation    of    a    bio-inspired tactile   module   that   comprises   a   32   taxels   array   and   a   9 DOF   MARG   (Magnetic,   Angular   Rate,   and   Gravity)   sensor,   a   flexible   compliant   structure   and   a   deep   pressure sensor.   Making   use   of   a   complementary   filter   the   orientation   of   the   shallow   sensors   can   be   estimated   and   the   force applied on such sensors is conducted through the compliant structure to the deep pressure sensor. Related publications: T.E.A.   Oliveira , A.-M.   Cretu   and   E.M.   Petriu,   “Design   of   a   Multi-Modal   Bio-Inspired Tactile   Module”,   submitted to IEEE Trans. Instrumentation and Measurement , June 2016, NEW . T.E.A.   Oliveira ,   V.P.   Fonseca,   A.-M.   Cretu,   E.M.   Petriu,   “Multi-Modal   Bio-Inspired   Tactile   Module”,   UOttawa Graduate    and    Research    Day ,    University    of    Ottawa,    Mar.    2016    ( Research    Poster    Prize    in    Electrical Engineering, first place; IEEE Research Poster Prize, first place ), NEW . This   project   aims   at   proposing   automated   solutions   for   3D   object   modeling   at multiple resolutions in the context of virtual reality. An   original   solution,   based   on   an   unsupervised   neural   network,    is   proposed to   guide   the   creation   of   selectively   densified   meshes.   A   neural   gas   network, adapts   its   nodes   during   training   to   capture   the   embedded   shape   of   the   object. Regions   of   interest   are   then   identified   as   areas   with   higher   density   of   nodes   in the      adapted      neural      gas      map. Meshes   at   different   level   of   detail   for an    object,    which    preserve    these regions   of   interest,   are   constructed by   adapting   a   classical   simplification algorithm.   The   simplification   process will   therefore   only   affect   the   regions of   lower   interest,   ensuring   that   the characteristics     of     an     object     are preserved   even   at   lower   resolutions. A   novel   solution   based   on   learning   is proposed   to   select   the   number   of   faces   for   the   discrete   models   of   an object   at   different   resolutions.   Finally,   selectively   densified   object   meshes are   incorporated   in   a   discrete   level-of-detail   method   for   presentation   in virtual reality applications. Another   track   of   the   project   aims   at   developing   an   original   application   of   biologically-inspired    visual    attention    for     improved    perception-based modeling   of   3D   objects.   In   an   initial   step,   an   adapted   computational   model of   visual   attention   is   used   to   identify   areas   of   interest   over   the   3D   shape   of   an   object.   Points   of   interest   are   then identified   as   the   centroids   of   these   salient   regions   and   integrated,   along   with   their   immediate   n -neighbors,   and   using a simplification algorithm, i nto a continuous distance-dependent level-of-details method. Support: Related publications: M.   Chagnon-Forget ,   A.-M.   Cretu,   and   S.   Bouchard,   "Visual-Attention   Based   Interest   Point   Detection   for Perceptually-Improved 3D Object Modeling", submitted to ACM Trans. Applied Perception , NEW . A.-M.   Cretu,   M.   Chagnon-Forget    and   P.   Payeur,   “Selectively   Densified   3D   Object   Modeling   Based   on Regions    of    Interest    Using    Neural    Gas    Networks”,    accepted    for    publication,    Soft    Computing,    2016,    doi: 10.1007/s00500-016-2132-z,   NEW . H.   Monette-Thériault,   A.-M.   Cretu,   and   P.   Payeur,   "3D   Object   Modeling   with   Neural   Gas   Based   Selective Densification   of   Surface   Meshes”,   IEEE   Int.   Conf.   Systems,   Man,   and   Cybernetics,    pp.   1373-1378,   San Diego, US, 2014. One    of    the    tracks    of    the    project    aims    at    the    development    a natural   gesture    interface   that   tracks   and   recognizes   in   real- time   static   and   dynamic   hand   gestures   of   a   user   based   on depth   data   collected   by   a   Kinect   sensor.   A   novel   algorithm   is proposed   to   improve   the   scanning   time   in   order   to   identify   the   first pixel   on   the   hand   contour   within   this   space.   Starting   from   this pixel,   a   directional   search   algorithm   allows   for   the   identification   of the    entire    hand    contour.    The    K-curvature    algorithm    is    then employed   to   locate   the   fingertips   over   the   contour,   and   dynamic time   warping   s   used   to   select   gesture   candidates   and   also   to recognize   gestures   by   comparing   an   observed   gesture   with   a series   of   pre-recorded   reference   gestures.   Two   possible   applications   of   this   work   are   discussed   and   evaluated:   one for   interpretation   of   sign   digits   and   popular   gestures   for   a   friendlier   human-machine   interaction,   the   other   one   for   the natural control of a software interface. Another   track   of   the   project   aims   at   the   development   of   a   system   capable   to control    the   arm   movement   of   a   robot   by   mimicking   the   gestures   of   an actor   captured   by   a   markerless   vision   sensor.   The   Kinect   for   Xbox   is   used   to recuperate   angle   information   at   the   level   of   the   actor's   arm   and   an   interaction module   transforms   it   into   a   usable   format   for   real-time   robot   arm   control.   To avoid   self-collisions,   the   distance   between   the   two   arms   is   computed   in   real- time   and   the   motion   is   not   executed   if   this   distance   becomes   smaller   the   twice the    diameter    of    the    member.    A    software    architecture    is    proposed    and implemented   for   this   purpose.   The   feasibility   of   our   approach   is   demonstrated on a NAO robot. Related publications:   G.   Plouffe ,   and   A.-M.Cretu,   “Static   and   Dynamic   Hand   Gesture   Recognition   in   Depth   Data   Using   Dynamic Time Warping“, IEEE Trans. Instrumentation and Measurement , vol. 65, no.2, pp. 305-316, Feb. 2016, NEW . G.   Plouffe ,   A.-M.   Cretu,   and   P.   Payeur,   “Natural   Human-Computer   Interaction   Using   Hand   Gestures”,   IEEE Symp. Haptic Audio-Visual Environments and Games , pp. 57-62, Ottawa, ON, Oct. 2015. S.   Filiatrault ,   and   A.-M.   Cretu,   "Human   Arm   Motion   Imitation   Using   a   Humanoid   Robot",   IEEE   Int.   Symp. Robotic and Sensors Environments , pp. 31-36, Timisoara, Romania, 2014 . This    project    explores    the    capabilities    of    Kinect    for    human    movement   quantification    for    in-home    physical    exercise    monitoring.   The    analysis    goes beyond    the    state-of-the-art    by    monitoring    more    joints    and    offering    more advanced   reporting   capabilities   on   the   movement   such   as:   the   position   and trajectory    of    each    joint,    the    working    envelope    of    each    body    member,    the average   velocity,   and   the   fatigue   of   the   user   after   an   exercise   sequence.   This data   can   be   visualised   and   compared   to   a   standard   (e.g.   a   healthy   user   for rehabilitation   purposes)   or   an   ideal   performance   (e.g.   perfect   sport   pose   for exercising)   in   order   to   give   the   user   a   measure   on   his/her   own   performance and   incite   his/her   motivation   to   continue   the   training   program.   Such   information can   be   used   as   well   by   a   therapist   or   professional   sports   trainer   to   evaluate   the progress   of   a   patient   or   of   a   trainee.    An   adaptation   is   proposed   as   well   for   the posture estimation in the co ntext of  piano pedagogy. Collaborators/ Support: Related publications: P.   Payeur,   G.   M.   Nascimento,   J.   Beacon,   G.   Comeau,   A.-M.   Cretu,   V.   D'Aoust,   and   M.-A.   Charpentier , "Human   Gesture   Quantification:   an   Evaluation Tool   for   Somatic Training   and   Piano   Peformance",   IEEE   Symp. Haptic Audio-Visual Environments and Games , pp. 100-105, Dallas, Texas, US, Oct. 2014. S.   Gauthier    and   A.-M.   Cretu,   "Human   Movement   Quantification   using   Kinect   for   In-Home   Physical   Exercise Monitoring”,   IEEE   Int.   Conf.   Computational   Intelligence   and   Virtual   Environments   for   Measurement   Systems and Applications , pp. 6-11, Ottawa, May 2014. Virtual    reality    has    already    been    successfully    used    as    a    therapeutic    tool    for    the   treatment    of    various    phobias.    Due    to    advances    in    the    3D    graphics    and    in    the computing   power,   the   real-time   visual   rendering   of   a   virtual   world   poses   no   significant problems   nowadays.   However,   the   haptic   interaction   with   such   environments   remains a    challenge.    This    paper    explores    the    haptic    interaction    with    a    dedicated    virtual environment   in   spider   phobia   treatment   to   elicit   disgust,   as   changes   in   fear   and   in disgust     were     shown     to     be     highly     associated     with     the     observed     decline     in arachnophobic    symptoms.    A    dedicated    virtual    environment    is    programmed    within which a Novint Falcon haptic device is used for the interaction with a virtual spider.   Collaborators: Related publications:   M.   Laforest,   S.   Bouchard,   A.-M.   Cretu,   and   O.   Mesly,   “Inducing   an   Anxiety   Response   Using   a   Contaminated Virtual   Environment:   Validation   of   a   Therapeutic   Tool   for   Obsessive-Compulsive   Disorder”,   submitted   to Frontiers in ICT (Information and Communication Technologies): Virtual Environments, Mar. 2016, NEW. M.   Cavrag,    G.   Larivière, A.-M.   Cretu,   and   S.   Bouchard,   "Interaction   with   Virtual   Spiders   for   Eliciting   Disgust   in the   Treatment   of   Phobias",   IEEE   Symp.   Haptic   Audio-Visual   Environments   and   Games ,   pp.   29-34,   Dallas, Texas, US, Oct. 2014. Recent    years    have    witnessed    an    increasing    interest    into    intelligent   autonomous    robots    able    to    execute    tasks    in    unknown    or    only    partially known   environments.   One   important   problem   to   deal   with   in   this   context   is the     capability     of     the     robot     to     autonomously     sense     its     immediate environment   and   understand   its   contents   in   order   to   be   able   to   navigate safely   and   perform   its   tasks   without   human   intervention.   The   objective   of this    project    is    to    extend    these    capabilities    by    a    fast    and    automated generation   of   an   environment   map,   associated   with   the   recognition   of   the objects   contained   in   it. These   will   serve   later   for   robot   path   planning   and   the execution of predetermined actions on the recognized objects. As   recent   research   has   demonstrated   that   depth   cues   have   a   considerable impact   on   human   fixations,   and   that   salient   maps   can   be   improved   by incorp orating   depth   information,   a   part   of   this   project   aims   at   the   exploitation of depth information to improve segmentation for salient object detection..  Collaborators/ Support: Related publications:   F.   Audet,   A.-M.   Cretu,   and   M.S. Allili,   “Salient   Object   Segmentation   in   RGB-D   Data”,   submitted   to   Int.   Symp. Visual Computing , Las Vegas, Dec. 2016,  NEW. The long term objective of this research project is to attempt to establish reliable associations between traffic data and accident occurrence. These associations should help a proactive approach for accident management by an automatic detection of prone locations and conditions for accidents. Collaborators/ Support: Related publications:   A.-M.   Cretu,   A.   Mageau-Pétrin   and   C.   Hopgood,   “Traffic   Accident   Modeling   and   Prediction”,   Ottawa   Police Services Research Project, Technical report, May. 2016, 33 pages, NEW. A.-M.   Cretu,   and   A.   Mageau-Pétrin,   “Intelligence-led   traffic   enforcement:   Data   mining   techniques   for   accident prediction”, Ottawa Police Services Research Project, Technical report, Jan. 2016, 159 slides, NEW. The   objective   of   this   project   is   to   study   the   impact   of   the   use   of   the   avatars   (or   profile   images)   for   the   detection   of   communities   in   social networks. Support :

This   

project   

investigates   

novel   

approaches   

for   

building

 

identification   

in   

aerial   

images,   

that   

combine   

classical

segmentation     

algorithms,     

     

user     

guided     

training

approaches and supervised learning solutions.

Collaborators/ Support: Related publications: J.     Tremblay-Gosselin      and     A.-M.     Cretu,     "A Supervised    Training    and    Learning    Method    for Building      Identification      in      Remotely      Sensed Images",    IEEE    Int.    Symp.    Robotic    and    Sensors Environments , Washington, US, 21-23 Oct. 2013, pp.73-78. A.-M.   Cretu,   and   P.   Payeur,   "Building   Detection   in   Aerial   Images   Based   on   Watershed   and   Visual   Attention Feature Descriptors", Canadian Conf.  Computer and Robot Vision , pp. 265-272, Regina, 2013. A.-M.   Cretu,   "Evolving   Sensor   System   Environments   with   Visual   Attention:   An   Experimental   Exploration", Proc.      IEEE   Symp.   Robotic   and   Sensors   Environments ,   pp.   97-102,   Magdeburg,   Germany,   16-18   Nov.   2012 (invited talk). This   project   explores   the   capabilities   of   computational   intelligence   for   fault   detection   in   sensor   networks.   Neural- network   approaches   are   exploited   in   order   to   provide   a   general   solution   covering   typical   sensor   faults   and   to   replace complex   sets   of   individual   detection   methods.   For   this   purpose,   an   appropriate   set   of   fault   relevant   features   is identified   in   a   first   step.   A   generic   neural-network   structure   and   learning   strategy   is   then   chosen   and   adapted   for detecting   multiple   fault   types.   Afterwards   the   approach   is   applied   on   a   common   used   sensor   system   and   evaluated with deterministi c fault injections. Collaborators: Related publications:   G.   Jager,   S.   Zug,   T.   Brade,   A.   Dietrich,   C.   Steup,   C.   Moewes,   A.-M.   Cretu,   “Assessing   Neural   Networks   for Sensor    Fault    Detection”,    IEEE    Int.    Conf.    Computational    Intelligence    and    Virtual    Environments    for Measurement Systems and Applications , pp. 70-75, Ottawa, May 2014 . Drawing    inspiration    from    the    significantly    superior performance   of   humans   to   extract   and   interpret   visual information,    the    proposed    research    transposes    the visual   attention   mechanisms   into   biologically-inspired computational    systems    to    develop    new    techniques and    computational    resources    capable    to    interpret complex   images   with   large   variations   in   their   content and   characteristics.   The   project   is   dedicated   to   the exploration,        implementation,        refinement        and validation      of      a      series      of      biologically-inspired computational     models     for     attention     and     object recognition    dedicated    to    specific    tasks,    such    as building detection in the context of geo-imaging. Collaborators/ Support: Related publications: A.-M.   Cretu,   and   P.   Payeur,   "Visual   Attention   Model   with   Context   Learning   for   Building   Detection   in   Satellite Images", Int. Journal Smart Sensing and Intelligent Systems , vol. 5, no.4, pp. 742-766, Dec. 2012 M.   I.   Sina ,   P.   Payeur,   and   A.-M.   Cretu,      "Object   Recognition   on   Satellite   Images   with   Biologically-Inspired Computational   Approaches",   IEEE   Int.   Symp.   Applied   Computational   Intelligence   and   Informatics ,   pp.   81-86, Timisoara, Romania, 24-26 May 2012. M.   I.   Sina ,   A.-M.   Cretu,   and   P.   Payeur,   “Biological   Visual   Attention   Guided   Automatic   Image   Segmentation with   Application   in   Satellite   Imaging”,   Proc.   IS&T/SPIE   Electronic   Imaging   Conference,   Human   Vision   and Electronic Imaging Track,  vol. 8291, Burlingame, California, USA, Jan. 2012. The   automated   servicing   of vehicles    is    becoming    more and   more   a   reality   in   today’s world.          While          certain operations,     such     as     car washing,     require     only     a rough   model   of   the   surface of        a        vehicle,        other operations,          such          as changing   of   a   wheel   or   filling the   gas   tank,   require   correct   localization   of   the   different   parts   of   the   vehicle   on   which   operations   are   to   be   performed. The   proposed   image-based   approach   to   roughly   localize   vehicle   parts   over   the   surface   of   a   vehicle   with   a   bounding box   approach   is   based   on   a   model   of   human   visual   attention.   The   proposed   method   is   automatically   adapted   for different   views   of   a   vehicle   and   obtains   average   localization   rates   for   different   vehicle   parts   of   over   95%   for   a   dataset of   120   vehicles   belonging   to   three   categories,   namely   sedan,   SUV   and   wagon   and   allows,   with   the   addition   of   the active   contour   models,   for   a   more   complete   and   accurate   description   of   vehicle   parts   contours   than   other   state-of- the-art solutions. Collaborators/ Support: Related publications: 5. D.   Nakhaeinia,   P.   Payeur,   A.   Chávez-Aragón,   A.-M.   Cretu,   R.   Laganière,   and   R.   Macknojia,   “Surface Following   with   an   RGB-D   Vision-Guided   Robotic   System   for   Vehicle   Security   Screening”,   Int.   Journal   Smart Sensing and Intelligent Systems, vol. 9, no. 2, pp. 419-447, Jun. 2016, NEW. A.-M.   Cretu,   P.   Payeur,   R.   Laganière,   "An Application   of   a   Bio-Inspired   Visual Attention   Model   Guided   for   the Localization of Vehicle Parts", Applied Soft Computing,  vol.31. pp. 369-380, 2015. R.    Fareh,    P.    Payeur,    D.    Nakhaeinia,    R.    Macknojia,    A.    Chávez-Aragón,    A.-M.    Cretu,    P.    Laferrière,    R. Laganière.   R.   Toledo,   "An   Integrated   Vision-Guided   Robotic   System   for   Rapid   Vehicle   Inspection",   IEEE   Int. Systems Conference , SysCon, Ottawa, Mar. 2014 ( Best Paper Award ). A.-M.   Cretu,   P.   Payeur,   "Image-Based   Localization   of   Vehicle   Parts   Guided   by   Visual   Attention",   Proc.   IEEE Int. Conf. Instrumentation and Measurement Technology , pp. 533-538, Graz, Austria, May 2012. The    continuous    rise    in    the    amount    of    vehicles    in circulation       brings       an       increasing       need       for automatically     and     efficiently     recognizing     vehicle categories      for      multiple      applications      such      as optimizing   available   parking   spaces,   balancing   ferry space,   perceiving   highway   toll,   planning   infrastructure and    managing    traffic,    or    servicing    vehicles.    This project   describes   the   design   and   implementation   of   a vehicle   classification   system   using   a   set   of   images collected   from   6   views.   The   proposed   computational system        combines        human        visual        attention mechanisms   to   identify   a   set   of   salient   discriminative features    and    a    series    of    binary    support    vector machines to achieve fast automated classification. Collaborators/ Support: Related publications: A.-M.    Cretu,    and    P.    Payeur,    "Biologically- Inspired     Visual     Attention     Features     for     a Vehicle   Classification Task",   Int.   Journal   Smart Sensing   and   Intelligent   Systems ,   vol.   4,   no.   3, pp. 402-423, Sep. 2011. A.-M.    Cretu,    P.    Payeur,    and    R.    Laganière, "Salient   Features   Based   on   Visual   Attention   for   Multi-View   Vehicle   Classification",   Proc.   IEEE   Int.   Conf. Computational   Intelligence   for   Measurement   Systems   and   Applications ,   pp.   64-69,   Ottawa,   Canada,   Sep. 2011 . This    project    addresses    the    problem    of    node    selection    using    computational intelligence   techniques   for   a   risk-aware   robotic   sensor   networks   applied   to   critical infrastructure   protection.   The   goal   is   to   maintain   a   secure   perimeter   around   a critical    infrastructure,    which    is    best    maintained    by    detecting    high-risk    network events   and   mitigate   them   through   a   response   involving   the   most   suitable   robotic   assets. Collaborators: Related publications: J.   McCausland,    R.   Abielmona,   R.   Falcon,   A.-M.   Cretu   and   E.M.   Petriu,   "On   the   Role   of   Multi-Objective Optimization    in    Risk    Mitigation    for    Critical    Infrastructures    with    Robotic    Sensor    Networks",    accepted    for publication,   ACM   Conference   Companion   on   Genetic   and   Evolutionary   Computation,    Vancouver,   Canada,   pp. 1269-1276,  July 2014, NEW . J.   McCausland ,   R.   Abielmona,   A.   -M.   Cretu,   Rafael   Falcon,   and   E.M.   Petriu,   “A   Proactive   Risk-Aware Robotic   Sensor   Network   for   Critical   Infrastructure   Protection”,   poster   session,   2013 Annual   Research   Review Healthcare Support through Information Technology Enhancements (hSITE) , Montreal, QC, 18 Nov. 2013,   J.   McCausland ,   R.   Abielmona,   R.   Falcon,   A.-M.   Cretu   and   E.M.   Petriu,   "Auction-Based   Node   Selection   of Optimal   and   Concurrent   Responses   for   a   Risk-Aware   Robotic   Sensor   Network",   IEEE   Int.   Symp.   Robotic   and Sensors Environments , Washington, US, 21-23 Oct. 2013, pp. 136-141 . The    project    discusses    the    design    and implementation        of        an        automated framework    that    provides    the    necessary information   to   the   controller   of   a   robotic hand    to    ensure    safe    model-based    3D deformable           object           manipulation. Measurements      corresponding      to      the interaction     force     at     the     level     of     the fingertips     and     to     the     position     of     the fingertips   of   a   three-fingered   robotic   hand are    associated    with    the    contours    of    a deformed   object   tracked   in   a   series   of   images   using   neural-network   approaches.   The   resulting   model   not   only captures   the   behavior   of   the   object   but   is   also   able   to   predict   its   behavior   for   previously   unseen   interactions   without any   assumption   on   the   object’s   material.   Such   models   allow   the   controller   of   the   robotic   hand   to   achieve   better controlled grasp and more elaborate manipulation capabilities. Related publications: A.-M.   Cretu,   P.   Payeur,   and   E.M.   Petriu,   “Soft   Object   Deformation   Monitoring   and   Learning   for   Model-Based Robotic Hand Manipulation”, IEEE Trans. Systems, Man and Cybernetics - Part B , vol. 42, no. 3, Jun. 2012. A.-M.   Cretu,   P.   Payeur,   and   E.M.   Petriu,   “Learning   and   Prediction   of   Soft   Object   Deformation   using   Visual Analysis   of   Robot   Interaction”,      Proc.   Int.   Symp.   Visual   Computing , ISVC2010,   Las   Vegas,   Nevada,   USA,   G. Bebis et al. (Eds), LNCS 6454, pp. 232-241, Springer, 2010.     F.   Khalil,   P.   Payeur, and   A.-M.   Cretu,   “Integrated   Multisensory   Robotic   Hand   System   for   Deformable   Object Manipulation”,   Proc.   Int.   Conf.   Robotics   and   Applications ,   pp.   159-166,   Cambridge,   Massachusetts,   US, 2010. A.-M.   Cretu,   P.   Payeur,   E.   M.   Petriu   and   F.   Khalil,   “Estimation   of   Deformable   Object   Properties   from   Visual Data   and   Robotic   Hand   Interaction   Measurements   for   Virtualized   Reality Applications   ”,   Proc.   IEEE   Int.   Symp. Haptic Audio Visual Environments and Their Applications , pp.168-173, Phoenix, AZ,  Oct. 2010.      A.-M.   Cretu,   P.   Payeur,   E.   M.   Petriu   and   F.   Khalil,   “Deformable   Object   Segmentation   and   Contour   Tracking   in Image   Sequences   Using   Unsupervised   Networks”,   Proc.   Canadian   Conf.   Computer   and   Robot   Vision ,   pp. 277-284, Ottawa, Canada,  May 2010. This   project   explores   some   aspects of    intelligent    sensing    for    advanced robotic    applications,    with    the    main objective     of     designing     innovative approaches    for    automatic    selection of    regions    of    observation    for    fixed and    mobile    sensors    to    collect    only relevant       measurements       without human     guidance.     The     proposed neural   gas   network   solution   selects regions      of      interest      for      further sampling    from    a    cloud    of    sparsely collected     3D     measurements.     The technique    automatically    determines bounded     areas     where     sensing     is required   at   higher   resolution   to   accurately   map   3D   surfaces.   Therefore   it   provides   significant   benefits   over   brute force strategies as scanning time is reduced and the size of the dataset is kept manageable. Collaborators/ Support: Related publications: P.   Payeur,   P.   Curtis,   A.-M.   Cretu,   "Computational   Methods   for   Selective   Acquisition   of   Depth   Measurements: an   Experimental   Evaluation",   Int.   Conf.      Advanced   Concepts   for   Intelligent   Vision   Systems ,   Poznan,   Poland, J. Blanc-Talon et al. (Eds.) LNCS 8192, pp. 389-401, 2013. P.   Payeur,   P.   Curtis, A.-M.   Cretu,   "Computational   Methods   for   Selective Acquisition   of   Depth   Measurements   in Machine Perception", IEEE Int. Conf. Systems, Man, and Cybernetics , Manchester, UK, pp.876-881, 2013. A.-M.   Cretu,   P.   Payeur,   and   E.M.   Petriu,   “Selective   Range   Data Acquisition   Driven   by   Neural   Gas   Networks”, IEEE Trans. Instrumentation and Measurement,  vol. 58, no. 6, pp. 2634-2642, 2009. A.-M. Cretu, P. Payeur, and E.M. Petriu, “Neural Gas and Growing Neural Gas Networks for Selective 3D Sensing: A Comparative Study”, Sensors & Transducers, vol. 5, pp. 119-134, 2009. A.-M.   Cretu,   P.   Payeur,   and   E.M.   Petriu,   “Selective   Tactile   Data   Acquisition   on   3D   Deformable   Objects   for Virtualized     Reality     Applications”,     Proc.     IEEE     Int.     Workshop     Computational     Intelligence     in     Virtual Environments , pp.14-19, Nashville, TN, USA, Apr. 2009 A.-M.   Cretu,   E.M.   Petriu,   and   P.   Payeur,   “Growing   Neural   Gas   Networks   for   Selective   3D   Scanning”,   Proc. IEEE Int. Workshop on Robotic and Sensors Environments , pp. 108-113, Ottawa, Canada, Oct. 2008. A.-M.   Cretu,   P.   Payeur,   and   E.M.   Petriu,   “Selective   Vision   Sensing   Based   on   Neural   Gas   Network”,   Proc. IEEE   Int.   Conf.   Instrumentation   and   Measurement   Technology,    pp.   478-483,   Vancouver,   Canada,   May   2008 (Best Student Paper Award).   This     work     presents     a     general-purpose scheme    for    measuring,    constructing    and representing       geometric       and       elastic behavior   of   deformable   objects   without   a priori    knowledge    on    the    shape    and    the material   that   the   objects   under   study   are made   of.   The   proposed   solution   is      based on   an   advantageous   combination   of   neural network   architectures   and   an   original   force- deformation    measurement    procedure.    An innovative      non-uniform      selective      data acquisition      algorithm      based      on      self- organizing     neural     architectures     (namely neural    gas    and    growing    neural    gas)    is developed    to    selectively    and    iteratively identify   regions   of   interest   and   guide   the acquisition   of   data   only   on   those   points   that   are   relevant   for   both   the   geometric   model   and   the   mapping   of   the   elastic behavior,   starting   from   a   sparse   point-cloud   of   an   object.   Multi-resolution   object   models   are   obtained   using   the   initial sparse   model   or   the   (growing   or)   neural   gas   map   if   a   more   compressed   model   is   desired,   and   augmenting   it   with   the higher   resolution   measurements   selectively   collected   over   the   regions   of   interest.   A   feedforward   neural   network   is then   employed   to   capture   the   complex   relationship   between   an   applied   force,   its   magnitude,   its   angle   of   application and   its   point   of   interaction,   the   object   pose and   the   deformation   stage   of   the   object   on one      side,      and      the      object      surface deformation    for    each    region    with    similar geometric    and    elastic    behavior    on    the other   side.   The   proposed   framework   works directly   from   raw   range   data   and   obtains compact   point-based   models.   It   can   deal with       different       types       of       materials, distinguishes   between   the   different   stages of    deformation    of    an    object    and    models homogeneous      and      non-homogeneous objects   as   well.   It   also   offers   the   desired degree of control to the user. Support: Related publication: A.M.   Cretu,   “Experimental   Data   Acquisition   and   Modeling   of   3D   Objects   using   Neural   Networks”,   Ph.D. Thesis, University of Ottawa , 2009. This    project    presents    a    critical    comparison between     three     neural     architectures     for     3D object     representation     in     terms     of     purpose, computational    cost,    complexity,    conformance and    convenience,    ease    of    manipulation    and potential    uses    in    the    context    of    virtualized reality.   Starting   from   a   pointcloud   that   embeds the    shape    of    the    object    to    be    modeled,    a volumetric    representation    is    obtained    using    a multilayered    feedforward    neural    network    or    a surface    representation    using    either    the    self- organizing   map   or   the   neural   gas   network.   The representation   provided   by   the   neural   networks is   simple,   compact   and   accurate.   The   models can   be   easily   transformed   in   size,   position   (affine   transformations)   and   shape   (deformation).   Some   potential   uses   of the   presented   architectures   in   the   context   of   virtualized   reality   are   for   the   modeling   of   set   operations   and   object morphing, for the detection of objects collision and for object recognition, object motion estimation and segmentation. Related publications: A.-M.   Cretu,   “Neural   Network   Modeling   of   3D   Objects   for   Virtualized   Reality   Applications”,   M.A.Sc.   Thesis, University of Ottawa , 2003. A.-M.    Cretu    and    E.M.    Petriu,    “Neural    Network-Based   Adaptive    Sampling    of    3D    Object    Surface    Elastic Properties”, IEEE Trans. Instrumentation and Measurement,  vol. 55, no. 2, pp. 483-492, 2006. A.-M.   Cretu,   E.M.   Petriu,   and   G.G.   Patry,   “Neural-Network   Based   Models   of   3D   Objects   for   Virtualized Reality:   A   Comparative   Study”,   IEEE   Trans.   Instrumentation   and   Measurement ,   vol.   55,   no.   1,   pp.   99-111, 2006. A.-M.   Cretu,   E.M.   Petriu,   and   G.G.   Patry,   “A   Comparison   of   Neural   Network   Architectures   for   the   Geometric Modeling   of   3D   Objects”,   Proc.   Conf.   Computational   Intelligence   for   Measurement   Systems   and Applications , pp. 155-160, Boston, USA, Jul. 2004. A.-M.   Cretu,   E.M.   Petriu,   and   G.G.Patry,   “Neural   Network   Architecture   for   3D   Object   Representation”,   Proc. IEEE   Int.   Workshop   on   Haptic,   Audio   and   Visual   Environments   and   Their   Applications ,   pp.   31-36,   Ottawa, Canada, Sep. 2003. Controlling    robotic    interventions    on    small    devices    creates important    challenges    on    the    sensing    stage    as    resolution limitations   of   non-contact   sensors   are   rapidly   reached.   The integration   of   haptic   sensors   to   refine   information   provided   by vision   sensors   appears   as   a   very   promising   approach   in   the development   of   autonomous   robotic   systems   as   it   reproduces the    multiplicity    of    sensing    sources    used    by    humans.    This project    proposes    an    intelligent    multimodal    sensor    system developed      to      enhance      the      haptic-control      of      robotic manipulations    of    small    3D    objects.    The    proposed    system combines    a    16x16    array    of    Force    Sensing    Resistor    (FSR) elements   to   refine   3D   shape   measurements   in   selected   areas previously    monitored    with    a    laser    range    finder.    Using    the integrated     technologies,     the     sensor     system     is     able     to recognize     small-size     objects     that     cannot     be     accurately differentiated   through   range   measurements   and   provides   an   estimate   of   the   objects   orientation.   Characteristics   of the   system   are   demonstrated   in   the   context   of   a   robotic   intervention   that   requires   fine   objects   to   be   localized   and identified for their shape and orientation. Related publications: P.    Payeur,    C.    Pasca,    A.-M.    Cretu,    and    E.M.    Petriu,    “Intelligent    Haptic    Sensor    System    for    Robotic Manipulation”, IEEE Trans. Instrumentation and Measurement,  vol. 54, no. 4, pp. 1583-1592, 2005. C.   Pasca,   P.   Payeur,   E.M.   Petriu,   A.-M.   Cretu,   “Intelligent   Haptic   Sensor   System   for   Robotic   Manipulation”, Proc. IEEE Int. Conf. Instrumentation and Measurement Technology , pp. 279-284, Italy, 2004. E.   M.   Petriu,   S.K.S.   Yeung,   S.R.   Das,   A.-M.   Cretu,   and   H.J.W.   Spoelder,   “Robotic   Tactile   Recognition   of Pseudo-Random   Encoded   Objects”,   IEEE   Trans.   Instrumentation   and   Measurement ,   vol.   53,   no.   5,   pp.   1425- 1432, 2004. The        inherent        complex nature     of     the     wastewater treatment   process,   the   lack of    proper    knowledge    and description   of   the   biological phenomena    and    the    large fluctuations    in    time    of    the numerous             parameters implied       (flow-rates       and nutrients     loadings)     makes the    automatic    control    of    a wastewater          plant          a complicated      and      difficult problem   to   tackle.   The   use of       online       sensors       for continuous   measurement   of wastewater     components     is prone   to   unpredictable   breakdowns.   Low   nutrient   conditions   in   a   wastewater   treatment   plant   may   cause   the   failure   of the   effective   nitrogen   and   phosphorus   removal   for   a   considerable   period.   This   project   proposes   neural-network modeling   approaches   for   the   model-based   control   of   a   wastewater   treatment   plant   in   terms   of   air   flow-rate   and municipal wastewater components. Related publications: K-Y.    Ko,    G.G.    Patry,    A.-M.    Cretu,    and    E.M.    Petriu,    “Neural    Network    Model    of    Municipal    Wastewater Components   for   a   Wastewater   Treatment   Plant”,   Proc.   IEEE   Int.   Workshop   on   Advanced   Environmental Sensing and Monitoring Techniques , pp. 23-28, Como, Italy, Jul. 2003. K.-Y.   Ko,   G.G.   Patry,   A.-M.   Cretu,   E.M.   Petriu,   “Neural   Network   Model   for   Wastewater   Treatment   Plant Control”,   Proc.   IEEE   Int.   Workshop   on   Soft   Computing   Techniques   in   Instrumentation,   Measurement   and Related Applications , pp. 38-43, May 2003.

Neuro-inspired computational intelligence for geo-imaging systems

Prospective Students
Students

Localization of Vehicle Parts Guided by Visual Attention

Salient Features for Image-Based Vehicle Classification

Risk-Aware Wireless Sensor Network for Critical Infrastructure Protection

Deformable Object Tracking and Modeling for Robotic Hand Manipulation 

Selective Range Data Acquisition

Experimental data acquisition and modeling of 3D deformable objects using neural networks

Neural network modeling of 3D objects for virtualized reality applications

Pattern classification and recognition from tactile data

Neural network models for wastewater treatment plant control

Computational intelligence for pattern recognition in aerial imaging

Computational intelligence in sensor environments

Perceptually-improved multi-resolution 3D object modeling

Community detection in social networks using avatars or images

3D environment mapping and object recognition in RGB-D data

Virtual environments for phobia treatment

Motion tracking and imitation for natural human-machine interaction

Courtesy of  the Cyberpsychology lab UQO

Human motion quantification for activity monitoring

Courtesy of  Larus Technologies

Multimodal biologically-inspired tactile sensing

Intelligence-led traffic enforcement