Automated Analysis of Multi-Modal Medical Data using Deep Belief Networks

Automated Analysis of Multi-Modal Medical Data using Deep Belief Networks

Recently, magnetic resonance and ultrasound imaging have found utility as adjuncts to mammography in the detection and management of breast cancer. This project will develop novel machine learning techniques that optimally integrate information from each of these data sources so as to improve the efficiency and accuracy of breast cancer diagnosis.

See details in Lean More, Project Page and About

Precision Radiology

Precision Radiology

In this work, we propose new prognostic methods that predict 5-year mortality in elderly individuals using chest computed tomography (CT). The methods consist of a classifier that performs this prediction using a set of features extracted from the CT image and segmentation maps of multiple anatomic structures. Preliminary results of this work have been published in ISBI 2017 and Scientific Reports (SREP). After the SREP publication, the work had a press release, which caught the attention of the general media: Fox News, Daily Mail, Wired, The Australian (Section Counting the Decimal Places), Huffington Post, ComputerWorld, The Lead, TechExplore, Medical News Today, Engadget, Inside South Australia, Indaily, The Advertiser, NewsX, Gizmodo (India). Check these interviews: Luke’s live radio interview (Radio Adelaide), Lyle’s Researchgate interview.

See details in Project Page


Started in 2014 we deployed a large scale 3D visual sensing infrastructure in Lisbon’s Airport. Currently with more than 30 depth cameras (3D sensors), became a key technology for queue management in   border control and security checkpoints. Fully developed at SIPg@LarSys was transferred to Thales Group that operates and maintains the infrastructure. It provides high-level analytics, extracting queue operating patterns from visual motion analysis and 3D tracking data. Hinging on sparse data representations and efficient convex optimization algorithms it subsumes part of our f

rontier research: i) low-rank and sparse models with missing data,  ii) large scale optimization algorithms and iii) 2D and 3D computer vision algorithms for tracking and

recognition. This research was published in top journals IEEE-PAMI, several IEEE-CVPR (highest ranked publication in Pattern Recognition) and summarized in IEEE-ITS ( It contributed strongly to the establishment of Thales’ innovation hub in Lisbon who currently sponsors several of our research projects for “smart-cities” (smartbyke, walkbot, smartcitysense)

LARSyS 2018 Annual Meeting

The 2018 LARSyS Annual Meeting was a success!

For two full days researchers from ISR, IN+, M-ITI and Maretec gathered at Pavilhão do Conhecimento to exchange knowledge and debate future challenges and opportunities. Take a look at some of what went on during all these vibrant activities by checking the LARSyS Flickr here or explore the recently live website of the Associated Laboratory, for more information.

See details in Link

Underwater Noise Monitoring

Acoustic Surveillance System

Underwater noise has been identified by EU as a pollutant for biological species, including marine mammals, fish, invertebrates and to biodiversity as a whole. Despite the vast amount of scientific theoretical, laboratorial and experimental work, the extent of impact of noise on these species is not clear but its monitoring is a first step for control. The SUBECO project aims at developing a prediction and monitoring system of the underwater noise off the coast of continental Portugal. This is a must have tool to support the mission of protecting the marine environment but also for security and defense.

See details in Project Page


A Context Aware and Video-Based Risk Descriptor for Cyclists

Monitoring cyclists’ data is a keystone to foster urban cyclists’ safety by helping urban planners to design safer cyclist routes. In this work, we propose a fully image-based framework to assess the route risk from the cyclist perspective. From smartphone sequences of images, we are able to automatically identify events considering different risk criteria based on the cyclist’s motion and object detection. This method provides context on the situation and is independent from the expertise level of the cyclist. From the inertial sensor data, we automatically detect the route segments performed by bicycle, applying behavior analysis techniques. We test our methods on real data, attaining very promising results in terms of risk classification and behavior analysis accuracy.

See details in Project Page

Discriminative Optimization

A Learned based method to design cost functions

DO is an innovative way of estimating a surrogate of the gradient of a “well behaved” cost function from data and solve a computer vision problems  involving models (e.g. error functions) shaped  by  training data. This approach faces two main challenges: (i) designing a cost function with a local optimum at an acceptable solution, and (ii) developing an efficient numerical method to search for one (or multiple) of these local optima. While designing such functions is feasible in the noiseless case, the stability and location of local optima are mostly unknown under noise, occlusion, or missing data. In practice, this can result in undesirable local optima or not having a local optimum in the expected place. On the other hand, numerical optimization algorithms in high-dimensional spaces are typically local and often rely on expensive first or second order information to guide the search. To overcome these limitations, this paper proposes Discriminative Optimization (DO), a method that learns search directions from data without the need of a cost function.

See details in Project Page

Vehicle Counting

Deep Understanding of Urban Traffic from Large-Scale City Cameras

The challenge is to count vehicles in a city-scale  low resolution, low frame rate network of urban cameras.  The target city is NY where 200+ cameras stream video from selected places. In this work several Deep Learning solutions are presented with unprecedented performance. In a very diverse conditions (sunny, cloudy, rainy) the deep-learning model is able to estimate the correct number of cars with errors of 1.5 cars (MAE).

See details in Project Page


Content and Ontology based Art Image Annotation and Retrieval

PrintArt is a project that gathers researchers from the Instituto Superior Tecnico, the Faculdade de Letras da Universidade de Lisboa, and the Museu Nacional do Azulejo with the purpose of designing a software to aid the study and the identification of Portuguese tile art.

Mainly due to the ease of reproduction and transportation, prints were used as the favoured means to make pictures and information available throughout the world. In this way, these art works quickly reached the hands of craftsman which used them as sources of inspiration, replicating them in different media, among which the tiles are particularly noteworthy. Trademark of Portuguese culture, the tiles have been produced continuously for five centuries, benefiting from the original prints in composition and theme, but using them freely; changing proportions, adding and removing figures, simplifying or enriching backgrounds, inverting figures, among other things.

See details in Project Page

IBM Scientific Awards

Pinar Oguz Ekim is the second woman to receive the IBM Scientific Award in 23 years. The award ceremony takes place this Tuesday in Lisbon.

The award, worth €15,000, “aims to distinguish the contribution of research work to the development of computer science and information technology,” IBM said in a statement.

The work of Pinar Ekim, from the Institute of Systems and Robotics (Instituto Superior Técnico), is entitled “Robust location algorithms in sensor networks with target tracking applications”, is based on the PhD thesis of the researcher and addresses” the problem for the determination of geographical positions of one or more agents (eg people, vehicles or animals) from measures of mutual distance and distances to reference points “.

According to the company, “this work introduces a new approach in how, in places where it is not possible to access the GPS system (either by equipment limitations or by the absence of GPS signals indoors), alternative technologies can be used to measure distances using acoustic signals, radio signals, or both”.

See details in Link