Abstract: We combine and extend the previous work on DCT-based image signatures and face detection to determine the visual saliency. To this end, we transfer the scalar definition of image signatures to quaternion images and thus introduce a novel saliency method using quaternion type-II DCT image signatures. Furthermore, we use MCT-based face detection to model the important influence of faces on the visual saliency using rotated elliptical Gaussian weight functions and evaluate several integration schemes. In order to demonstrate the performance of the proposed methods, we evaluate our approach on the Bruce-Tsotsos (Toronto) and Cerf (FIFA) benchmark eye-tracking data sets. Additionally, we present evaluation results on the Bruce-Tsotsos data set of the most important spectral saliency approaches. We achieve state-of-the-art results in terms of the well-established area under curve (AUC) measure on the Bruce-Tsotsos data set and come close to the ideal AUC on the Cerf data set - with less than one millisecond to calculate the bottom-up QDCT saliency map.

Keywords: Spectral Saliency, Quaternion, DCT Image Signatures; MCT Face Detection; Attention; Human Gaze, Eye-Tracking

Download: [pdf] [bibtex] [code #1 - saliency] [code #2 - Matlab AUC measure implementation] [data set #1 - Cerf/FIFA face detections]

Abstract: We present a unified approach to improve the localization and the perception of a robot in a new environment by using already installed cameras. Using our approach we are able to localize arbitrary cameras in multi-camera environments while automatically extending the camera network in an online, unattended, real-time way. This way, all cameras can be used to improve the perception of the scene, and additional cameras can be added in real-time, e.g., to remove blind spots. [...] we use it to iteratively calibrate the camera network as well as to localize arbitrary cameras, e.g. of mobile phones or robots, inside a multi-camera environment. [...]

Keywords: Camera Pose Estimation, Camera Calibration; Scale Ambiguity; SIFT; Multi-Camera Environment, Smart Room, Robot Localization

Download: [pdf] [bibtex] [code #1] [data set #1]

Abstract: Multimodal attention is a key requirement for humanoid robots in order to navigate in complex environments and act as social, cognitive human partners. To this end, robots have to incorporate attention mechanisms that focus the processing on the potentially most relevant stimuli while controlling the sensor orientation to improve the perception of these stimuli. In this paper, we present our implementation of audio-visual saliency-based attention that we integrated in a system for knowledge-driven audio-visual scene analysis and object-based world modeling. For this purpose, we introduce a novel isophote-based method for proto-object segmentation of saliency maps, a surprise-based auditory saliency definition, and a parametric 3-D model for multimodal saliency fusion. The applicability of the proposed system is demonstrated in a series of experiments.

Keywords: Multimodal, Audio-Visual Attention; Auditory Surprise; Isophote-based Visual Proto-Objects; Parametric 3-D Saliency Model and Fusion; Object-based Inhibition of Return; Object-based Scene Exploration and Hierarchical Analysis

Download: [pdf] [bibtex] [code #1 - visual saliency] [code #2 - windowed Gaussian surprise / auditory saliency]

Abstract: In recent years, natural verbal and non-verbal human-robot interaction has attracted an increasing interest. Therefore, models for robustly detecting and describing visual attributes of objects such as, e.g., colors are of great importance. However, in order to learn robust models of visual attributes, large data sets are required. Based on the idea to overcome the shortage of annotated training data by acquiring images from the Internet, we propose a method for robustly learning natural color models. Its novel aspects with respect to prior art are: firstly, a randomized HSL transformation that reflects the slight variations and noise of colors observed in real-world imaging sensors; secondly, a probabilistic ranking and selection of the training samples, which removes a considerable amount of outliers from the training data. [...]

Keywords: Color Terms; Color Naming; Web-based Learning; Natural vs. Web-based Image Statisticsi; Domain Adaptation; Probabilistic HSL Model; Probabilistic Latent Semantic Analysis (pLSA); Human-Robot Interaction

Download: [pdf] [bibtex] [data set #1 - Google-512] [report]

Abstract: Identifying verbally and non-verbally referred-to objects is an important aspect of human-robot interaction. Most importantly, it is essential to achieve a joint focus of attention and, thus, a natural interaction behavior. In this contribution, we introduce a saliency-based model that reflects how multi-modal referring acts influence the visual search, i.e. the task to find a specific object in a scene. Therefore, we combine positional information obtained from pointing gestures with contextual knowledge about the visual appearance of the referred-to object obtained from language. The available information is then integrated into a biologically-motivated saliency model that forms the basis for visual search. We prove the feasibility of the proposed approach by presenting the results of an experimental evaluation.

Keywords: Modulatable Neuron-based, Phase-based, Spectral Whitening Saliency; Attention; Visual Search; Objects; Color; Shared Attention, Joint Attention; Multi-Modal Interaction, Gestures, Pointing, Language; Deictic Interaction; Spoken Human-Robot Interaction

Download: [pdf] [bibtex]

Abstract: When persons interact, non-verbal cues are used to direct the attention of persons towards objects of interest. Achieving joint attention this way is an important aspect of natural communication. Most importantly, it allows to couple verbal descriptions with the visual appearance of objects, if the referred-to object is non-verbally indicated. In this contribution, we present a system that utilizes bottom-up saliency and pointing gestures to efficiently identify pointed-at objects. Furthermore, the system focuses the visual attention by steering a pan-tilt-zoom camera towards the object of interest and thus provides a suitable model-view for SIFT-based recognition and learning.

Keywords: Spectral Residual Saliency, Spectral Whitening Saliency; Joint/Shared Attention, Pointing Gestures; Object Detection and Learning; Maximally Stable Extremal Regions (MSER); Scale-Invariant Feature Transform (SIFT); Active Pan-Tilt-Zoom Camera; Human-Robot Interaction

Download: [pdf] [bibtex] [errata]

Abstract: This paper considers the problem of multi-modal saliency and attention. Saliency is a cue that is often used for directing attention of a computer vision system, e.g., in smart environments or for robots. Unlike the majority of recent publications on visual/audio saliency, we aim at a well grounded integration of several modalities. The proposed framework is based on fuzzy aggregations and offers a flexible, plausible, and efficient way for combining multi-modal saliency information. Besides incorporating different modalities, we extend classical 2D saliency maps to multi-camera and multi-modal 3D saliency spaces. For experimental validation we realized the proposed system within a smart environment. The evaluation took place for a demanding setup under real-life conditions, including focus of attention selection for multiple subjects and concurrently active modalities.

Keywords: Multi-Camera; 3-D Spatial Saliency; Multi-Modal Saliency; Attention; Active Multi-Camera Control; Volumetric Intersection, Minimal Reconstruction Error; View Selection, Viewpoint Selection; Multi-Modal Sensor Fusion; Fuzzy; Smart Room; Human-Machine Interaction

Download: [pdf] [bibtex] [code #1 - reconstruction error approximation]

Abstract: Focusing their attention to the most relevant information is a fundamental biological concept, which allows humans to (re-)act rapidly and safely in complex and unfamiliar environments. This principle has successfully been adopted for technical systems where sensory stimuli need to be processed in an efficient and robust way. In this paper a multi-modal attention system for smart environments is described that explicitly respects efficiency and robustness aspects already by its architecture. The system facilitates unconstrained human-machine interaction by integrating multiple sensory information of different modalities.

Keywords: Multi-Modal; Multi-Camera; 3-D Spatial Saliency; Attention; Active Multi-Camera Control; Volumetric Intersection; View Selection, Viewpoint Selection; Real-Time Performance; Distributed, Scalable System; Design; Smart Environment, Smart Room; Human-Machine Interaction

Download: [pdf] [bibtex]

Abstract: Intelligent environments are supposed to simplify the everyday life of their users. To reach this target, a multitude of sensors is necessary to create a model of the current scene. The complete processing of the resulting sensor data stream can exceed the available processing capacities and inhibit the real-time processing of the complete sensor data. A possible solution to this problem is a fast pre-selection of potentially relevant sensor data and the restriction of complex calculations on the pre-selected sensor data. That leaves the question of what is potentially relevant. [...]

Download: [bibtex]

Abstract: In Petersen's well-known cubic graph every vortex is missed by some longest cycle. Thomassen produced a planar graph with this property. Grünbaum found a cubic graph, in which any two vertices are missed by some longest cycle. In this paper we present a cubic planar graph fulfilling this condition.

Download: [pdf] [bibtex]

Download: [bibtex]

Download: [bibtex]

Download: [bibtex]