Results of automated detection of complex patterns in temporal data, such as trajectories of moving objects, may be not good enough due to the use of strict pattern specifications derived from imprecise domain concepts. To address this challenge, we propose a novel visual analytics approach that combines expert knowledge and automated pattern detection results to construct features that effectively distinguish patterns of interest from other types of behaviour. These features are then used to create interactive visualisations enabling a human analyst to generate labelled examples for building a feature-based pattern classifier. We evaluate our approach through a case study focused on detecting trawling activities in fishing vessel trajectories, demonstrating significant improvements in pattern recognition by leveraging domain knowledge and incorporating human reasoning and feedback. Our contribution is a novel framework that integrates human expertise and analytical reasoning with ML or AI techniques, advancing the field of data analytics.

Towards a Hybrid Human-Machine Discovery of Complex Movement Patterns

Volunteered Geographic Information in the form of actively and passively generated spatial content offers great potential to study people's activities, emotional perceptions, and mobility behavior. Realizing this potential requires methods which take into account the specific properties of such data, for example, its heterogeneity, subjectivity, and spatial resolution but also temporal relevance and bias.

Extraction and Visually Driven Analysis of VGI for

Extraction and Visually Driven Analysis of VGI for Understanding People's Behavior in Relation to Multifaceted Context

Human-in-the-Loop

Human-in-the-Loop: Visual Analytics for Building Models Recognising Behavioural Patterns in Time Series

Dense and complex air traffic requires higher levels of automation than those exhibited by tactical conflict detection and resolution (CD&R) tools that air traffic controllers (ATCOs) use today: AI tools can act on their own initiative, increasing the capacity of ATCOs to control higher volumes of traffic. However, given that the air traffic control (ATC) domain is safety critical, requires AI systems to which ATCOs are comfortable to relinquishing control, guaranteeing operational integrity and automation adoption. Two major factors towards this goal are quality of solutions and operational transparency. ResoLver, the system that this article presents, addresses these challenges using an enhanced graph convolutional reinforcement learning method operating in a multiagent setting where each agent – representing a flight – performs a CD&R task, jointly with other agents. We show that ResoLver can provide high-quality solutions with respect to stakeholders interests (air traffic controllers and airspace users), addressing also operational transparency issues, which have been validated by ATCOs in simulated real-world settings.

Deep reinforcement learning in service of air traffic controllers to resolve tactical conflicts

We propose an approach to exploring interrelationships between two or more sequences of events when events occurring in one sequence both affect and are affected by events occurring in another sequences. We present the approach by example of exploring the dynamic relationships between COVID pandemic events and changes in population mobility behaviours across various countries. The key idea is to generate data capturing the temporal context of each event, i.e., what types of events occurred in different sequences within a specified time buffer around this event. An application of 2D space embedding to the context data reveals groups of events occurring in similar contexts. We can investigate the types of events each group consists of and see when and where these events and these contexts took place. By interactive or algorithmic clustering of the context data, we categorise event contexts based on their similarities, which allows us to compute, visualise, explore, and compare summary statistics of the context clusters, as well as exploring their distribution over time and other data dimensions.

Exploring Relationships between Events in Context

Mobility data captures the locations of moving objects such as humans, animals, and cars. With the availability of GPS-equipped mobile devices and other inexpensive location-tracking technologies, mobility data is collected ubiquitously. In recent years, the use of mobility data has demonstrated significant impact in various domains including traffic management, urban planning, and health sciences. In this paper, we present the domain of mobility data science. Towards a unified approach to mobility data science, we present a pipeline having the following components: mobility data collection, cleaning, analysis, management, and privacy. For each of these components, we explain how mobility data science differs from general data science, we survey the current state of the art, and describe open challenges for the research community in the coming years.

Mobility Data Science

Mobility Data Science: Perspectives and Challenges

We present a novel approach to understanding space use by moving entities based on repeated patterns of place visits and transitions. Our approach represents trajectories as text documents consisting of sequences of place visits or transitions and applies topic modelling to the corpus of these documents. The resulting topics represent combinations of places or transitions, respectively, that repeatedly co-occur in trips. Visualisation of the results in the spatial context reveals the regions of place connectivity through movements and the major channels used to traverse the space. This enables understanding of the use of space as a medium for movement. We compare the possibilities provided by topic modelling to alternative approaches exploiting a numeric measure of pairwise connectedness. We have extensively explored the potential of utilising topic modelling by applying our approach to multiple real-world movement data sets with different data collection procedures and varying spatial and temporal properties: GPS road traffic of cars, unconstrained movement on a football pitch, and episodic movement data reflecting social media posting events. The approach successfully demonstrated the ability to uncover meaningful patterns and interesting insights. We thoroughly discuss different aspects of the approach and share the knowledge and experience we have gained with people who might be potentially interested in analysing movement data by means of topic modelling methods.

Topic modelling for spatial insights

Topic modelling for spatial insights: Uncovering space use from movement data

Spatio-temporal event sequences consist of activities or occurrences involving various interconnected elements in space and time. Exploring these sequences with topic modeling is a relatively new and evolving research area. We use topic modeling to analyze football games, as an example of complex and under-explored spatio-temporal event data. A key challenge in topic modeling is selecting the most suitable number of topics for the downstream application. Selecting too few topics oversimplifies the data, merging distinct patterns, whereas selecting too many can fragment coherent themes into overlapping categories. We propose a visual analytics technique that uses dimensionality reduction on topics derived from multiple topic modeling runs, each with a different number of topics. Our technique organizes all the topics in a hierarchical layout based on their spatial similarity, making it easier to make an informed decision about selecting the most expressive set of topics that represent distinctive spatial patterns.     We apply our visual analytics technique to a football dataset, illustrating how it can be used to select an appropriate set of topics for this data. We then use these topics to represent game episodes, which help us summarize game dynamics and uncover insights into the games.

Interplay of Visual Analytics and Topic Modeling in Gameplay Analysis


We propose an approach to exploring interrelationships between two or more sequences of events when events occurring in one sequence both affect and are affected by events occurring in another sequences. We present the approach by example of exploring the dynamic relationships between COVID pandemic events and changes in population mobility behaviours across various countries. The key idea is to generate data capturing the temporal context of each event, i.e., what types of events occurred in different sequences within a specified time buffer around this event. An application of 2D space embedding to the context data reveals groups of events occurring in similar contexts. We can investigate the types of events each group consists of and see when and where these events and these contexts took place. By interactive or algorithmic clustering of the context data, we categorise event contexts based on their similarities, which allows us to compute, visualise, explore, and compare summary statistics of the context clusters, as well as exploring their distribution over time and other data dimensions. 

For a set of geographically referenced time series, we generate a matrix arrangement with rows corresponding to linearized geographical positions and columns to time steps. This matrix is treated as a background map suitable for application of common methods for thematic data visualization. 2D spatial positions are transformed to 1D ordering of the matrix rows by means of one of existing methods for dimensionality reduction. We illustrate the approach by representing the dynamics of the COVID-19 pandemic and population mobility levels throughout the provinces of Spain.

Creating Quasi-Maps for Uncovering Space-Time Patterns

The COVID-19 pandemic has prompted a surge in research efforts to understand its transmission dynamics and societal impact. In this study, we focus on Spain, a nation significantly affected by the pandemic, analyzing relationships between daily movement within and between its 52 provinces (Ponce-de-Leon et al 2021) alongside daily counts of COVID-19 active cases. By employing lagged correlation and Granger causality analyses, we aim to unveil the temporal relationships between mobility and disease spread, shedding light on crucial factors influencing the pandemic's trajectory. Such insights help to inform targeted interventions and public health strategies aimed at curbing the spread of infectious diseases.

Mapping Time Lags in Epidemics

This paper presents a novel approach to the problem of time periodization, which involves dividing the time span of a complex dynamic phenomenon into periods that enclose different relatively stable states or development trends. The challenge lies in finding such a division of the time that takes into account diverse behaviours of multiple components of the phenomenon while being simple and easy to interpret. Despite the importance of this problem, it has not received sufficient attention in the fields of visual analytics and data science. We use a real-world example from aviation and an additional usage scenario on analysing mobility trends during the COVID-19 pandemic to develop and test an analytical workflow that combines computational and interactive visual techniques. We highlight the differences between the two cases and show how they affect the use of different techniques. Through our investigation of possible variations in the time periodization problem, we discuss the potential of our approach to be used in various applications. Our contributions include defining and investigating an earlier neglected problem type, developing a practical and reproducible approach to solving problems of this type, and uncovering potential for formalization and development of computational methods.

It’s about Time: Analytical Time Periodization

We present a novel approach to analyze spatio-temporal movement patterns using topic modeling. Our approach represents trajectories as sequences of place visits and moves, applies topic modeling separately to each collection of sequences, and synthesizes results. This supports the identification of dominant topics for both place visits and moves, the exploration of spatial and temporal patterns of movement, enabling understanding of space use. The approach is applied to two real-world data sets of car movements in Milan and UK road traffic, demonstrating the ability to uncover meaningful patterns and insights.

Extracting Movement-based Topics for Analysis of S

Extracting Movement-based Topics for Analysis of Space Use

We present a conceptual framework for the development of visual interactive techniques to formalize and externalize trust in machine learning (ML) workflows. Currently, trust in ML applications is an implicit process that takes place in the user’s mind. As such, there is no method of feedback or communication of trust that can be acted upon. Our framework will be instrumental in developing interactive visualization approaches that will help users to efficiently and effectively build and communicate trust in ways that fit each of the ML process stages. We formulate several research questions and directions that include: 1) a typology/taxonomy of trust objects, trust issues, and possible reasons for (mis) trust; 2) formalisms to represent trust in machine-readable form; 3) means by which users can express their state of trust by interacting with a computer system (e.g., text, drawing, marking); 4) ways in which a system can facilitate users’ expression and communication of the state of trust; and 5) creation of visual interactive techniques for representation and exploration of trust over all stages of an ML pipeline.

The Flow of Trust

The Flow of Trust: A Visualization Framework to Externalize, Explore, and Explain Trust in ML Applications

Trustworthy machine learning requires a high level of interpretability of machine learning models, yet many models are inherently black-boxes. Training interpretable models instead---or using them to mimic the black-box model---seems like a viable solution. In practice, however, these interpretable models are still unintelligible due to their size and complexity. In this paper, we present an approach to explain the logic of large interpretable models that can be represented as sets of logical rules by a simple, and thus intelligible, descriptive model. The coarseness of this descriptive model and its fidelity to the original model can be controlled, so that a user can understand the original model in varying levels of depth. We showcase and discuss this approach on three real-world problems from healthcare, material science, and finance.

Re-interpreting rules interpretability

We introduce a concept of episode referring to a time interval in the development of a dynamic phenomenon that is characterized by multiple time-variant attributes. A data structure representing a single episode is a multivariate time series. To analyse collections of episodes, we propose an approach that is based on recognition of particular patterns in the temporal variation of the variables within episodes. Each episode is thus represented by a combination of patterns. Using this representation, we apply visual analytics techniques to fulfil a set of analysis tasks, such as investigation of the temporal distribution of the patterns, frequencies of transitions between the patterns in episode sequences, and co-occurrences of patterns of different variables within same episodes. We demonstrate our approach on two examples using real-world data, namely, dynamics of human mobility indicators during the COVID-19 pandemic and characteristics of football team movements during episodes of ball turnover.

Identifying, exploring, and interpreting time seri

Identifying, exploring, and interpreting time series shapes in multivariate time intervals

With the objective to enhance human performance and maximize engagement during the performance of tasks, we aim to advance automation for decision making in complex and large-scale multi-agent settings. Towards these goals, this paper presents a deep multi agent reinforcement learning method for resolving demand - capacity imbalances in real-world Air Traffic Management settings with thousands of agents. Agents comprising the system are able to jointly decide on the measures to be applied to resolve imbalances, while they provide explanations on their decisions: This information is rendered and explored via appropriate visual analytics tools. The paper presents how major challenges of scalability and complexity are addressed, and provides results from evaluation tests that show the abilities of models to provide high-quality solutions and high-fidelity explanations.

Explaining deep reinforcement learning decisions

Explaining deep reinforcement learning decisions in complex multiagent settings: towards enabling automation in air traffic flow management

 ABSTRACTDashboards are an increasingly popular form of data visualization. Large, complex, and dynamic mobility data present a number of challenges in dashboard design. The overall aim for dashboard design is to improve information communication and decision making, though big mobility data in particular require considering privacy alongside size and complexity. Taking these issues into account, a gap remains between wrangling mobility data and developing meaningful dashboard output. Therefore, there is a need for a framework that bridges this gap to support the mobility dashboard development and design process. In this paper we outline a conceptual framework for mobility data dashboards that provides guidance for the development process while considering mobility data structure, volume, complexity, varied application contexts, and privacy constraints. We illustrate the proposed framework’s components and process using example mobility dashboards with varied inputs, end-users and objectives. Overall, the framework offers a basis for developers to understand how informational displays of big mobility data are determined by end-user needs as well as the types of data selection, transformation, and display available to particular mobility datasets. 

Conceptual framework for developing dashboards

A conceptual framework for developing dashboards for big mobility data

Abstract The term ‘episode’ refers to a time interval in the development of a dynamic process or behaviour of an entity. Episode-based data consist of a set of episodes that are described using time series of multiple attribute values. Our research problem involves analysing episode-based data in order to understand the distribution of multi-attribute dynamic characteristics across a set of episodes. To solve this problem, we applied an existing theoretical model and developed a general approach that involves incrementally increasing data abstraction. We instantiated this general approach in an analysis procedure in which the value variation of each attribute within an episode is represented by a combination of symbols treated as a ‘word’. The variation of multiple attributes is thus represented by a combination of ‘words’ treated as a ‘text’. In this way, the the set of episodes is transformed to a collection of text documents. Topic modelling techniques applied to this collection find groups of related (i.e. repeatedly co-occurring) ‘words’, which are called ‘topics’. Given that the ‘words’ encode variation patterns of individual attributes, the ‘topics’ represent patterns of joint variation of multiple attributes. In the following steps, analysts interpret the topics and examine their distribution across all episodes using interactive visualizations. We test the effectiveness of the procedure by applying it to two types of episode-based data with distinct properties and introduce a range of generic and data type-specific visualization techniques that can support the interpretation and exploration of topic distribution.

Episodes and Topics in Multivariate Temporal Data

Dense and complex air traffic requires higher levels of automation than those exhibited by tactical conflict detection and resolution (CD&R) tools that air traffic controllers (ATCOs) use today: AI tools can act on their own initiative, increasing the capacity of ATCOs to control higher volumes of traffic. However, given that the air traffic control (ATC) domain is safety critical, requires AI systems to which ATCOs are comfortable to relinquishing control, guaranteeing operational integrity and automation adoption. Two major factors towards this goal are quality of solutions and operational transparency. ResoLver, the system that this article presents, addresses these challenges using an enhanced graph convolutional reinforcement learning method operating in a multiagent setting where each agent - representing a flight- performs a CD&R task, jointly with other agents. We show that ResoLver can provide high-quality solutions with respect to stakeholders interests (air traffic controllers and airspace users), addressing also operational transparency issues, which have been validated by ATCOs in simulated real-world settings.

Deep reinforcement learning in service of air traf

This paper introduces an approach to analysing multivariate time series (MVTS) data through progressive temporal abstraction of the data into patterns characterizing behavior of the studied dynamic phenomenon. The paper focuses on two core challenges: identifying basic behavior patterns of individual attributes and examining the temporal relations between these patterns across the range of attributes to derive higher-level abstractions of multi-attribute behavior. The proposed approach combines existing methods for univariate pattern extraction, computation of temporal relations according to the Allen’s time interval algebra, visual displays of the temporal relations, and interactive query operations into a cohesive visual analytics workflow. The paper describes application of the approach to real-world examples of population mobility data during the COVID-19 pandemic and characteristics of episodes in a football match, illustrating its versatility and effectiveness in understanding composite patterns of interrelated attribute behaviors in MVTS data.

Exploring and visualizing temporal relations in mu

Exploring and visualizing temporal relations in multivariate time series

We consider the general problem known as job shop scheduling, in which multiple jobs consist of sequential operations that need to be executed or served by appropriate machines having limited capacities. For example, train journeys (jobs) consist of moves and stops (operations) to be served by rail tracks and stations (machines). A schedule is an assignment of the job operations to machines and times where and when they will be executed. The developers of computational methods for job scheduling need tools enabling them to explore how their methods work. At a high level of generality, we define the system of pertinent exploration tasks and a combination of visualizations capable of supporting the tasks. We provide general descriptions of the purposes, contents, visual encoding, properties, and interactive facilities of the visualizations and illustrate them with images from an example implementation in air traffic management. We justify the design of the visualizations based on the tasks, principles of creating visualizations for pattern discovery, and scalability requirements. The outcomes of our research are sufficiently general to be of use in a variety of applications.

Supporting Visual Exploration of Iterative Job Sch

Supporting Visual Exploration of Iterative Job Scheduling

We introduce a new research area in visual analytics (VA) aiming to bridge existing gaps between methods of interactive machine learning (ML) and eXplainable Artificial Intelligence (XAI), on one side, and human minds, on the other side. The gaps are, first, a conceptual mismatch between ML/XAI outputs and human mental models and ways of reasoning, and second, a mismatch between the information quantity and level of detail and human capabilities to perceive and understand. A grand challenge is to adapt ML and XAI to human goals, concepts, values, and ways of thinking. Complementing the current efforts in XAI towards solving this challenge, VA can contribute by exploiting the potential of visualization as an effective way of communicating information to humans and a strong trigger of human abstractive perception and thinking. We propose a cross-disciplinary research framework and formulate research directions for VA.

Visual Analytics for Human-Centered Machine Learni

Visual Analytics for Human-Centered Machine Learning

Currently, the methodological and technical developments in visual analytics, as well as the existing theories, are not sufficiently grounded by empirical studies that can provide an understanding of the processes of visual data analysis, analytical reasoning and derivation of new knowledge by humans. We conducted an exploratory empirical study in which participants analysed complex and data-rich visualisations by detecting salient visual patterns, translating them into conceptual information structures and reasoning about those structures to construct an overall understanding of the analysis subject. Eye tracking and voice recording were used to capture this process. We analysed how the data we had collected match several existing theoretical models intended to describe visualisation-supported reasoning, knowledge building, decision making or use and development of mental models. We found that none of these theoretical models alone is sufficient for describing the processes of visual analysis and knowledge generation that we observed in our experiments, whereas a combination of three particular models could be apposite. We also pondered whether empirical studies like ours can be used to derive implications and recommendations for possible ways to support users of visual analytics systems. Our approaches to designing and conducting the experiments and analysing the empirical data were appropriate to the goals of the study and can be recommended for use in other empirical studies in visual analytics.

Seeking Patterns of Visual Pattern Discovery for K

Seeking Patterns of Visual Pattern Discovery for Knowledge Building

The word `pattern' frequently appears in the visualisation and
               visual analytics literature, but what do we mean when we talk
               about patterns? We propose a practicable definition of the
               concept of a pattern in a data distribution as a combination of
               multiple interrelated elements of two or more data components
               that can be represented and treated as a unified whole. Our
               theoretical model describes how patterns are made by
               relationships existing between data elements. Knowing the types
               of these relationships, it is possible to predict what kinds of
               patterns may exist. We demonstrate how our model underpins and
               refines the established fundamental principles of visualisation.
               The model also suggests a range of interactive analytical
               operations that can support visual analytics workflows where
               patterns, once discovered, are explicitly involved in further
               data analysis.

A theoretical model for pattern discovery in visua

A theoretical model for pattern discovery in visual analytics

Visual analytics techniques support the process of data analysis, reasoning, and knowledge building performed by a human analyst. The techniques combine interactive, human-controllable visual displays with interactive operations for data querying and filtering, data transformations, calculation of derived data, and application of computational techniques for analysis and modelling. We demonstrate the use of visual analytics techniques and procedures for analyzing Automatic Identification System (AIS) data. We begin with showing how visual analytics approaches can help in exploring properties of the data, detecting problems, and finding ways to clean and improve the data. Then we describe two analysis scenarios focusing on the events of vessel stopping and on the vessel traffic through the strait between the bay of Brest, France, and the outer sea. Thereby we show how different techniques are applied and combined.

Visual Analytics of Vessel Movement

Visual analytics science develops principles and methods for efficient human--computer collaboration in solving complex problems. Visual and interactive techniques are used to create conditions in which human analysts can effectively utilize their unique capabilities: the power of seeing, interpreting, linking, and reasoning. Visual analytics research deals with various types of data and analysis tasks from numerous application domains. A prominent research topic is analysis of spatiotemporal data, which may describe events occurring at different spatial locations, changes of attribute values associated with places or spatial objects, or movements of people, vehicles, or other objects. Such kinds of data are abundant in urban applications. Movement data are a quintessential type of spatiotemporal data because they can be considered from multiple perspectives as trajectories, as spatial events, and as changes of space-related attribute values. By example of movement data, we demonstrate the utilization of visual analytics techniques and approaches in data exploration and analysis.

Characterizing Mobility Aspects of Urban Context

Visual Analytics for Characterizing Mobility Aspects of Urban Context

Visualizing big and complex multivariate data is challenging. To address this challenge, we propose flexible visual analytics (FVA) with the aim to mitigate visual complexity and interaction complexity challenges in visual analytics, while maintaining the strengths of multiple perspectives on the studied data. At the heart of our proposed approach are transitions that fluidly transform data between user-relevant views to offer various perspectives and insights into the data. While smooth display transitions have been already proposed, there has not yet been an interdisciplinary discussion to systematically conceptualize and formalize these ideas. As a call to further action, we argue that future research is necessary to develop a conceptual framework for flexible visual analytics. We discuss preliminary ideas for prioritizing multi-aspect visual representations and multi-aspect transitions between them, and consider the display user for whom such depictions are produced and made available for visual analytics. With this contribution we aim to further facilitate visual analytics on complex data sets for varying data exploration tasks and purposes based on different user characteristics and data use contexts.

Toward flexible visual analytics augmented through

Toward flexible visual analytics augmented through smooth display transitions

A possible objective in analyzing trajectories of multiple simultaneously moving objects, such as football players during a game, is to extract and understand the general patterns of coordinated movement in different classes of situations as they develop. For achieving this objective, we propose an approach that includes a combination of query techniques for flexible selection of episodes of situation development, a method for dynamic aggregation of data from selected groups of episodes, and a data structure for representing the aggregates that enables their exploration and use in further analysis. The aggregation, which is meant to abstract general movement patterns, involves construction of new time-homomorphic reference systems owing to iterative application of aggregation operators to a sequence of data selections. As similar patterns may occur at different spatial locations, we also propose constructing new spatial reference systems for aligning and matching movements irrespective of their absolute locations. The approach was tested in application to tracking data from two Bundesliga games of the 2018/2019 season. It enabled detection of interesting and meaningful general patterns of team behaviors in three classes of situations defined by football experts. The experts found the approach and the underlying concepts worth implementing in tools for football analysts.

Constructing Spaces and Times for Tactical Analysi

Constructing Spaces and Times for Tactical Analysis in Football

In various domains, there are abundant streams or sequences of multi-item data of various kinds, e.g. streams of news and social media texts, sequences of genes and sports events, etc. Comparison is an important and general task in data analysis. For comparing data streams involving multiple items (e.g., words in texts, actors or action types in action sequences, visited places in itineraries, etc.), we propose Co-Bridges, a visual design involving connection and comparison techniques that reveal similarities and differences between two streams. Co-Bridges use river and bridge metaphors, where two sides of a river represent data streams, and bridges connect temporally or sequentially aligned segments of streams. Commonalities and differences between these segments in terms of involvement of various items are shown on the bridges. Interactive query tools support the selection of particular stream subsets for focused exploration. The visualization supports both qualitative (common and distinct items) and quantitative (stream volume, amount of item involvement) comparisons. We further propose Comparison-of-Comparisons, in which two or more Co-Bridges corresponding to different selections are juxtaposed. We test the applicability of the Co-Bridges in different domains, including social media text streams and sports event sequences. We perform an evaluation of the users' capability to understand and use Co-Bridges. The results confirm that Co-Bridges is effective for supporting pair-wise visual comparisons in a wide range of applications.

Co-Bridges: Pair-wise Visual Connection and Compar

Co-Bridges: Pair-wise Visual Connection and Comparison for Multi-item Data Streams

 ABSTRACTData classification, i.e. organising data items in groups (classes), is a general technique widely used in data visualisation and cartography, in particular, for creation of choropleth maps. Conventionally, data are classified by dividing the data range into intervals and assigning the same symbol or colour to all data falling within an interval. For instance, the intervals may be of the same length or may include the same number of data items. We propose a method for defining intervals so that some quantity represented by values of another attribute is equally distributed among the classes. This kind of classification supports exploratory analysis of relationships between the attribute used for the classification and the distribution of the phenomenon whose quantity is represented by the additional attribute. The approach may be especially useful when the distribution of the phenomenon is very unequal, with many data items having zero or low quantities and quite a few items having larger quantities. With such a distribution, standard statistical analysis of the relationships may be problematic. We demonstrate the potential of the approach by analysing data referring to a set of spatially distributed people (patients) in relationship to characteristics of the areas in which the people live. 

equal-distribution data classification

Automating and utilising equal-distribution data classification

The exponential increase in the availability of large-scale mobility data has fueled the vision of smart cities that will transform our lives. The truth is that we have just scratched the surface of the research challenges that should be tackled in order to make this vision a reality. Consequently, there is an increasing interest among different research communities (ranging from civil engineering to computer science) and industrial stakeholders in building knowledge discovery pipelines over such data sources. At the same time, this widespread data availability also raises privacy issues that must be considered by both industrial and academic stakeholders. In this paper, we provide a wide perspective on the role that big data have in reshaping cities. The paper covers the main aspects of urban data analytics, focusing on privacy issues, algorithms, applications and services, and georeferenced data from social media. In discussing these aspects, we leverage, as concrete examples and case studies of urban data science tools, the results obtained in the ``City of Citizens'' thematic area of the Horizon 2020 SoBigData initiative, which includes a virtual research environment with mobility datasets and urban analytics methods developed by several institutions around Europe. We conclude the paper outlining the main research challenges that urban data science has yet to address in order to help make the smart city vision a reality.

(So) Big Data and the transformation of the city

How can big data help to understand the migration phenomenon? In this paper, we try to answer this question through an analysis of various phases of migration, comparing traditional and novel data sources and models at each phase. We concentrate on three phases of migration, at each phase describing the state of the art and recent developments and ideas. The first phase includes the journey, and we study migration flows and stocks, providing examples where big data can have an impact. The second phase discusses the stay, i.e. migrant integration in the destination country. We explore various data sets and models that can be used to quantify and understand migrant integration, with the final aim of providing the basis for the construction of a novel multi-level integration index. The last phase is related to the effects of migration on the source countries and the return of migrants.

Human migration: the big data perspective

Visual analytics is a research discipline that is based on acknowledging the power and the necessity of the human vision, understanding, and reasoning in data analysis and problem solving. It develops a methodology of analysis that facilitates human activities by means of interactive visual representations of information. By examples from the domains of aviation and maritime transportation, we demonstrate the essence of the visual analytics methods and their utility for investigating properties of available data and analysing data for understanding real-world phenomena and deriving valuable knowledge. We describe four case studies in which distinct kinds of knowledge have been derived from trajectories of vessels and airplanes and related spatial and temporal data by human analytical reasoning empowered by interactive visual interfaces combined with computational operations.

VA in the Aviation and Maritime Domains

Visual Analytics in the Aviation and Maritime Domains

  This textbook presents the main principles of visual analytics and describes techniques and approaches that have proven their utility and can be readily reproduced. Special emphasis is placed on various instructive examples of analyses, in which the need for and the use of visualisations are explained in detail.  The book begins by introducing the main ideas and concepts of visual analytics and explaining why it should be considered an essential part of data science methodology and practices. It then describes the general principles underlying the visual analytics approaches, including those on appropriate visual representation, the use of interactive techniques, and classes of computational methods. It continues with discussing how to use visualisations for getting aware of data properties that need to be taken into account and for detecting possible data quality issues that may impair the analysis. The second part of the book describes visual analytics methods and workflows, organised by various data types including multidimensional data, data with spatial and temporal components, data describing binary relationships, texts, images and video. For each data type, the specific properties and issues are explained, the relevant analysis tasks are discussed, and appropriate methods and procedures are introduced. The focus here is not on the micro-level details of how the methods work, but on how the methods can be used and how they can be applied to data. The limitations of the methods are also discussed and possible pitfalls are identified.  The textbook is intended for students in data science and, more generally, anyone doing or planning to do practical data analysis. It includes numerous examples demonstrating how visual analytics techniques are used and how they can help analysts to understand the properties of data, gain insights into the subject reflected in the data, and build good models that can be trusted. Based on several years of teaching related courses at the City, University of London, the University of Bonn and TU Munich, as well as industry training at the Fraunhofer Institute IAIS and numerous summer schools, the main content is complemented by sample datasets and detailed, illustrated descriptions of exercises to practice applying visual analytics methods and workflows.

Visual analytics for data scientists

Visual analytics is a research discipline that is based on acknowledging the power and the necessity of the human vision, understanding, and reasoning in data analysis and problem solving. Visual analytics develops methods, analytical workflows, and software tools for analysing data of various types, particularly, spatio-temporal data, which can describe the processes going on in the environment, society, and economy. We briefly overview the achievements of the visual analytics research concerning spatio-temporal data analysis and discuss the major open problems.

STVA: a vision for 2020s

Spatio-temporal visual analytics: a vision for 2020s

Abstract Guidance is an emerging topic in the field of visual analytics. Guidance can support users in pursuing their analytical goals more efficiently and help in making the analysis successful. However, it is not clear how guidance approaches should be designed and what specific factors should be considered for effective support. In this paper, we approach this problem from the perspective of guidance designers. We present a framework comprising requirements and a set of specific phases designers should go through when designing guidance for visual analytics. We relate this process with a set of quality criteria we aim to support with our framework, that are necessary for obtaining a suitable and effective guidance solution. To demonstrate the practical usability of our methodology, we apply our framework to the design of guidance in three analysis scenarios and a design walk-through session. Moreover, we list the emerging challenges and report how the framework can be used to design guidance solutions that mitigate these issues.

Guide Me in Analysis

Guide Me in Analysis: A Framework for Guidance Designers

A method for citywide traffic analysis is introduced based on the combination of visual and analytical approaches. Large volumes of GPS data collected from urban vehicles are utilized. In the method, a traffic condition map is constructed, composed of five different layers featuring traffic conditions, road linkage, travel patterns, congestion zones, and traffic flows, respectively. Based on the map, specific transport situations surrounding the congested areas are examined and ways of reducing congestion are suggested. The method is evaluated in the aggregated metropolitan area of Athens and Piraeus in Greece, and the potential and the effectiveness of this technique in analysing traffic are demonstrated. With more and more urban vehicles being equipped with GPS devices, the method can be easily transferable to other regions, paving the way for the adoption of the approach for an up-to-date, spatial-temporal sensitive, visual and analytic method for traffic monitoring that supports the establishment of a more sustainable urban transportation system.

Citywide Traffic Analysis Based on the Combination

Citywide Traffic Analysis Based on the Combination of Visual and Analytic Approaches

The concept of place connectedness (traditionally termed `accessibility') refers to the ability of people to reach various services and to participate in activities. Connectedness by public transport is especially important for underprivileged and elderly people while the active use of public transport by the general population contributes in reducing traffic congestions and air pollution in cities. Place connectedness analyses are performed for a variety of purposes. In communication with transportation experts, we performed the conceptual modeling of the domain of problems related to place connectedness, defined the system of analysis tasks, and matched the tasks to visual analytics techniques that are capable to support them. In this paper, we introduce the task typology and present the visual analytics techniques using several example scenarios of place connectedness analyses.

VA of Place Connectedness by Public Transport

Visual Analysis of Place Connectedness by Public Transport

We define behavior as a set of actions performed by some actor during a period of time. We consider the problem of analyzing a large collection of behaviors by multiple actors, more specifically, identifying typical behaviors and spotting anomalous behaviors. We propose an approach leveraging topic modeling techniques - LDA (Latent Dirichlet Allocation) Ensembles - to represent categories of typical behaviors by topics that are obtained through topic modeling a behavior collection. When such methods are applied to text in natural languages, the quality of the extracted topics are usually judged based on the semantic relatedness of the terms pertinent to the topics. This criterion, however, is not necessarily applicable to topics extracted from non-textual data, such as action sets, since relationships between actions may not be obvious. We have developed a suite of visual and interactive techniques supporting the construction of an appropriate combination of topics based on other criteria, such as distinctiveness and coverage of the behavior set. Two case studies on analyzing operation behaviors in the security management system and visiting behaviors in an amusement park, and the expert evaluation of the first case study demonstrate the effectiveness of our approach.

LDA Ensembles for Interactive Exploration and Cate

LDA Ensembles for Interactive Exploration and Categorization of Behaviors

Visual analytics usually deals with complex data and uses sophisticated algorithmic, visual, and interactive techniques supporting the analysis. Findings and results of the analysis often need to be communicated to an audience that lacks visual analytics expertise. This requires analysis outcomes to be presented in simpler ways than that are typically used in visual analytics systems. However, not only analytical visualizations may be too complex for target audiences but also the information that needs to be presented. Analysis results may consist of multiple components, which may involve multiple heterogeneous facets. Hence, there exists a gap on the path from obtaining analysis findings to communicating them, within which two main challenges lie: information complexity and display complexity. We address this problem by proposing a general framework where data analysis and result presentation are linked by story synthesis, in which the analyst creates and organises story contents. Unlike previous research, where analytic findings are represented by stored display states, we treat findings as data constructs. We focus on selecting, assembling and organizing findings for further presentation rather than on tracking analysis history and enabling dual (i.e., explorative and communicative) use of data displays. In story synthesis, findings are selected, assembled, and arranged in meaningful layouts that take into account the structure of information and inherent properties of its components. We propose a workflow for applying the proposed conceptual framework in designing visual analytics systems and demonstrate the generality of the approach by applying it to two diverse domains, social media and movement analysis.

Supporting Story Synthesis: Bridging the Gap betwe

Supporting Story Synthesis: Bridging the Gap between Visual Analytics and Storytelling

We propose an approach to analyzing data in which texts are associated with spatial and temporal references with the aim to understand how the text semantics vary over space and time. To represent the semantics, we apply probabilistic topic modeling. After extracting a set of topics and representing the texts by vectors of topic weights, we aggregate the data into a data cube with the dimensions corresponding to the set of topics, the set of spatial locations (e.g., regions), and the time divided into suitable intervals according to the scale of the planned analysis. Each cube cell corresponds to a combination (topic, location, time interval) and contains aggregate measures characterizing the subset of the texts concerning this topic and having the spatial and temporal references within these location and interval. Based on this structure, we systematically describe the space of analysis tasks on exploring the interrelationships among the three heterogeneous information facets, semantics, space, and time. We introduce the operations of projecting and slicing the cube, which are used to decompose complex tasks into simpler subtasks. We then present a design of a visual analytics system intended to support these subtasks. To reduce the complexity of the user interface, we apply the principles of structural, visual, and operational uniformity while respecting the specific properties of each facet. The aggregated data are represented in three parallel views corresponding to the three facets and providing different complementary perspectives on the data. The views have similar look-and-feel to the extent allowed by the facet specifics. Uniform interactive operations applicable to any view support establishing links between the facets. The uniformity principle is also applied in supporting the projecting and slicing operations on the data cube. We evaluate the feasibility and utility of the approach by applying it in two analysis scenarios using geolocated social media data for studying people's reactions to social and natural events of different spatial and temporal scales.

Semantics-Space-Time Cube: A Conceptual Framework 

Semantics-Space-Time Cube: A Conceptual Framework for Systematic Analysis of Texts in Space and Time

User behaviour analytics (UBA) systems offer sophisticated models that capture users' behaviour over time with an aim to identify fraudulent activities that do not match their profiles. Motivated by the challenges in the interpretation of UBA models, this paper presents a visual analytics approach to help analysts gain a comprehensive understanding of user behaviour at multiple levels, namely individual and group level. We take a user-centred approach to design a visual analytics framework supporting the analysis of collections of users and the numerous sessions of activities they conduct within digital applications. The framework is centred around the concept of hierarchical user profiles that are built based on features derived from sessions, as well as on user tasks extracted using a topic modelling approach to summarise and stratify user behaviour. We externalise a series of analysis goals and tasks, and evaluate our methods through use cases conducted with experts. We observe that with the aid of interactive visual hierarchical user profiles, analysts are able to conduct exploratory and investigative analysis effectively, and able to understand the characteristics of user behaviour to make informed decisions whilst evaluating suspicious users and activities.

VASABI: Hierarchical User Profiles for Interactive

VASABI: Hierarchical User Profiles for Interactive Visual User Behaviour Analytics

As the number of moving objects increases, the challenges for achieving operational goals w.r.t. the mobility in many domains that are critical to economy and safety emerge dramatically. In domains such as air traffic management, this dictates a shift of operations' paradigm from location based, as it is today, to trajectory based, where trajectories are turned into ``first-class citizens''. Additionally, the increasing amount of data from heterogenous and disparate data sources implies the need for advanced analysis methods that require exploiting spatio-temporal mobility data in appropriate forms and at varying levels of abstraction. All these call for an in-principle way for organising integrated views of mobility data, with trajectories playing the main role. In this paper, we propose an ontology for modelling semantic trajectories, integrating spatio-temporal information regarding mobility of objects, at multiple, interlinked levels of abstraction. Our work builds upon a comprehensive framework that identifies fundamental spatio-temporal data types and specific conversions among these types. We validate the ontological specifications towards satisfying the needs of visual analysis tasks in the complex air traffic management domain, using real-world data.

The datAcron Ontology for the Specification of Sem

The datAcron Ontology for the Specification of Semantic Trajectories

 ABSTRACTEvents are a core concept of spatial information, but location-based social media (LBSM) provide information on reactions to events. Individuals have varied degrees of agency in initiating, reacting to or modifying the course of events, and reactions include observations of occurrence, expressions containing sentiment or emotions, or a call to action. Key characteristics of reactions include referent events and information about who reacted, when, where and how. Collective reactions are composed of multiple individual reactions sharing common referents. They can be characterized according to the following dimensions: spatial, temporal, social, thematic and interlinkage. We present a conceptual framework, which allows characterization and comparison of collective reactions. For a thematically well-defined class of event such as storms, we can explore differences and similarities in collective attribution of meaning across space and time. Other events may have very complex spatio-temporal signatures (e.g. political processes such as Brexit or elections), which can be decomposed into series of individual events (e.g. a temporal window around the result of a vote). The purpose of our framework is to explore ways in which collective reactions to events in LBSM can be described and underpin the development of methods for analysing and understanding collective reactions to events. 

A conceptual framework for studying collective rea

A conceptual framework for studying collective reactions to events in location-based social media

Action sequences, where atomic user actions are represented in a labelled, timestamped form, are becoming a fundamental data asset in the inspection and monitoring of user behaviour in digital systems. Although the analysis of such sequences is highly critical to the investigation of activities in cyber security applications, existing solutions fail to provide a comprehensive understanding due to the complex semantic and temporal characteristics of these data. This paper presents a visual analytics approach that aims to facilitate a user-involved, multi-faceted decision making process during the identification and the investigation of “unusual” action sequences. We first report the results of the task analysis and domain characterisation process. Then we describe the components of our multi-level analysis approach that comprises of constraint-based sequential pattern mining and semantic distance based clustering, and multi-scalar visualisations of users and their sequences. Finally, we demonstrate the applicability of our approach through a case study that involves tasks requiring effective decision-making by a group of domain experts. Although our solution here is tightly informed by a user-centred, domain-focused design process, we present findings and techniques that are transferable to other applications where the analysis of such sequences is of interest.

Understanding User Behaviour through Action Sequen

Understanding User Behaviour through Action Sequences: From the Usual to the Unusual

Spatial time series is a common type of data dealt with in many domains, such as economic statistics and environmental science. There have been many studies focusing on finding and analyzing various kinds of events in time series; the term `event' refers to significant changes or occurrences of particular patterns formed by consecutive attribute values. We focus on a further step in event analysis: discover temporal relationship patterns between event locations, i.e., repeated cases when there is a specific temporal relationship (same time, before, or after) between events occurring at two locations. This can provide important clues for understanding the formation and spreading mechanisms of events and interdependencies among spatial locations. We propose a visual exploration framework COPE (Co-Occurrence Pattern Exploration), which allows users to extract events of interest from data and detect various co-occurrence patterns among them. Case studies and expert reviews were conducted to verify the effectiveness and scalability of COPE using two real-world datasets.

COPE: Interactive Exploration of Co-Occurrence Pat

COPE: Interactive Exploration of Co-Occurrence Patterns in Spatial Time Series

Transportation agencies have an opportunity to leverage increasingly available trajectory datasets to improve their analyses and decision-making processes. However, this data is typically purchased from vendors, which means agencies must understand its potential benefits beforehand in order to properly assess its value relative to the cost of acquisition. While the literature concerned with trajectory data is rich, it is naturally fragmented and focused on technical contributions in niche areas, which makes it difficult for government agencies to assess its value across different transportation domains. To overcome this issue, this paper explores trajectory data from the perspective of a road transportation agency interested in acquiring trajectories to enhance its analysis. This paper provides a literature review illustrating applications of trajectory data in six areas of road transportation systems analysis: demand estimation, modeling human behavior, designing public transit, traffic performance measurement and prediction, environment, and safety. In addition, it visually explores 20 million GPS traces in Maryland, USA, illustrating the existing and suggesting new applications of trajectory data.

Applications of Trajectory Data From the Perspecti

Applications of Trajectory Data From the Perspective of a Road Transportation Agency: Literature Review and Maryland Case Study

In movement data analysis, there exists a problem of comparing multiple trajectories of moving objects to common or distinct reference trajectories. We introduce a general conceptual framework for comparative analysis of trajectories and an analytical procedure, which consists of (1) finding corresponding points in pairs of trajectories, (2) computation of pairwise difference measures, and (3) interactive visual analysis of the distributions of the differences with respect to space, time, set of moving objects, trajectory structures, and spatio-temporal context. We propose a combination of visualisation, interaction, and data transformation techniques supporting the analysis and demonstrate the use of our approach for solving a challenging problem from the aviation domain.

Analysis of Flight Variability: a Systematic Appro

Analysis of Flight Variability: a Systematic Approach

For understanding the circumstances, causes, and consequences of events that may happen during movement (e.g., harsh brake, sharp turn), it is necessary to analyze event context. The context includes dynamic attributes of the moving objects before and after the event and external context elements such as other moving objects, weather, terrain, etc. To explore events in context, we propose an analytical workflow including event contextualization, context pattern detection, and exploration of the spatio-temporal distribution of the detected patterns. The approach involves clustering of events based on the similarity of their contexts and interactive visual techniques for exploration of the distribution of the clusters in time, geographic space, and multidimensional attribute space. In close collaboration with domain experts, we apply our method to real-world vehicle trajectories with the purpose of identifying and investigating potentially dangerous driving behaviors.

Contextualized Analysis of Movement Events

While conventional applications for spatiotemporal datasets mostly focus on the relation between movers and environment, research questions in the analysis of collective movement typically focus more on relationships and dynamics between the moving entities themselves. Instead of concentrating on origin, destination and the way in between, this inter-mover perspective on spatiotemporal data allows to explain how moving groups are coordinating. Yet, only few visualization and Visual Analytics approaches focus on the relationships between movers. To illuminate this research gap, we propose initial steps towards a comprehensive formalization of coordination in collective movement based on temporal autocorrelation of distance matrices derived from basic movement characteristics. We exemplify how patterns can be encoded using autocorrelation cubes and outline the next steps towards an exhaustive formalization of coordination patterns.

Moving Together: Towards a Formalization of Collec

Moving Together: Towards a Formalization of Collective Movement

We present a visual analytical approach to exploring variation of topic popularity in social media (such as Twitter) over space and time. Our approach includes an analytical pipeline and a multi-view visualization tool. As attempts of topic extraction from very short texts like tweets may not produce meaningful results, we aggregate the texts prior to applying topic modelling techniques. Interactive visualisations support detection of burst events in social media posting activities at different locations, show the spatial, temporal, quantitative, and semantic aspects of these events, and enable the user to explore how popularity of topics varies over cities and time. A case study has been conducted using a real-world tweet dataset.

Visual Exploration of Spatial and Temporal Variati

Visual Exploration of Spatial and Temporal Variations of Tweet Topic Popularity

We propose an approach to interactive visual exploration of trajectories of moving objects in which trajectories are mapped onto different coordinate systems enabling the analyst to look at different aspects of the movement. Geographic visualization techniques can be applied to these coordinate systems in the same way as in usual geographic map displays.

Maps of Artificial Spaces to Explore Trajectories

Creating Maps of Artificial Spaces to Explore Trajectories

In this paper, we present an efficient in-DBMS framework for progressive time-aware sub-trajectory cluster analysis. In particular, we address two variants of the problem: (a) spatiotemporal sub-trajectory clustering and (b) index-based time-aware clustering at querying environment. Our approach for (a) relies on a two-phase process: a voting-and-segmentation phase followed by a sampling-and-clustering phase. Regarding (b), we organize data into partitions that correspond to groups of sub-trajectories, which are incrementally maintained in a hierarchical structure. Both approaches have been implemented in Hermes@PostgreSQL, a real Moving Object Database engine built on top of PostgreSQL, enabling users to perform progressive cluster analysis via simple SQL. The framework is also extended with a Visual Analytics (VA) tool to facilitate real world analysis

Time-Aware Sub-Trajectory Clustering

Time-Aware Sub-Trajectory Clustering in Hermes at PostgreSQL

Data quality management, especially data cleansing, has been extensively studied for many years in the areas of data management and visual analytics. In the paper, we first review and explore the relevant work from the research areas of data management, visual analytics and human-computer interaction. Then for different types of data such as multimedia data, textual data, trajectory data, and graph data, we summarize the common methods for improving data quality by leveraging data cleansing techniques at different analysis stages. Based on a thorough analysis, we propose a general visual analytics framework for interactively cleansing data. Finally, the challenges and opportunities are analyzed and discussed in the context of data and humans.

Steering data quality with visual analytics: The c

Steering data quality with visual analytics: The complexity challenge

In this paper, we list the goals for and the pros and cons of guidance, and we discuss the role that it can play not only in key low-level visualization tasks but also the more sophisticated model-generation tasks of visual analytics. Recent advances in artificial intelligence, particularly in machine learning, have led to high hopes regarding the possibilities of using automatic techniques to perform some of the tasks that are currently done manually using visualization by data analysts. However, visual analytics remains a complex activity, combining many different subtasks. Some of these tasks are relatively low-level, and it is clear how automation could play a role—for example, classification and clustering of data. Other tasks are much more abstract and require significant human creativity, for example, linking insights gleaned from a variety of disparate and heterogeneous data artifacts to build support for decision making. In this paper, we outline the potential applications of guidance, as well as the inputs to guidance. We discuss challenges in implementing guidance, including the inputs to guidance systems and how to provide guidance to users. We propose potential methods for evaluating the quality of guidance at different phases in the analytic process and introduce the potential negative effects of guidance as a source of bias in analytic decision making.

Guidance in the human–machine analytics process

Abstract To complement the currently existing definitions and conceptual frameworks of visual analytics, which focus mainly on activities performed by analysts and types of techniques they use, we attempt to define the expected results of these activities. We argue that the main goal of doing visual analytics is to build a mental and/or formal model of a certain piece of reality reflected in data. The purpose of the model may be to understand, to forecast or to control this piece of reality. Based on this model-building perspective, we propose a detailed conceptual framework in which the visual analytics process is considered as a goal-oriented workflow producing a model as a result. We demonstrate how this framework can be used for performing an analytical survey of the visual analytics research field and identifying the directions and areas where further research is needed.

VA as Model Building

Viewing Visual Analytics as Model Building

Clustering of trajectories of moving objects by similarity is an important technique in movement analysis. Existing distance functions assess the similarity between trajectories based on properties of the trajectory points or segments. The properties may include the spatial positions, times, and thematic attributes. There may be a need to focus the analysis on certain parts of trajectories, i.e., points and segments that have particular properties. According to the analysis focus, the analyst may need to cluster trajectories by similarity of their relevant parts only. Throughout the analysis process, the focus may change, and different parts of trajectories may become relevant. We propose an analytical workflow in which interactive filtering tools are used to attach relevance flags to elements of trajectories, clustering is done using a distance function that ignores irrelevant elements, and the resulting clusters are summarized for further analysis. We demonstrate how this workflow can be useful for different analysis tasks in three case studies with real data from the domain of air traffic. We propose a suite of generic techniques and visualization guidelines to support movement data analysis by means of relevance-aware trajectory clustering.

Clustering Trajectories by Relevant Parts for Air 

Clustering Trajectories by Relevant Parts for Air Traffic Analysis

Analysts in professional team sport regularly perform analysis to gain strategic and tactical insights into player and team behavior. Goals of team sport analysis regularly include identification of weaknesses of opposing teams, or assessing performance and improvement potential of a coached team. Current analysis workflows are typically based on the analysis of team videos. Also, analysts can rely on techniques from Information Visualization, to depict e.g., player or ball trajectories. However, video analysis is typically a time-consuming process, where the analyst needs to memorize and annotate scenes. In contrast, visualization typically relies on an abstract data model, often using abstract visual mappings, and is not directly linked to the observed movement context anymore. We propose a visual analytics system that tightly integrates team sport video recordings with abstract visualization of underlying trajectory data. We apply appropriate computer vision techniques to extract trajectory data from video input. Furthermore, we apply advanced trajectory and movement analysis techniques to derive relevant team sport analytic measures for region, event and player analysis in the case of soccer analysis. Our system seamlessly integrates video and visualization modalities, enabling analysts to draw on the advantages of both analysis forms. Several expert studies conducted with team sport analysts indicate the effectiveness of our integrated approach.

Bring It to the Pitch

Bring It to the Pitch: Combining Video and Movement Data to Enhance Team Sport Analysis

 A behaviour can be defined as a sequence of states or activities occurring one after another. A behaviour consisting of a finite number of reoccurring states/activities may be represented by a directed weighted graph with nodes and edges corresponding, respectively, to the possible states and transitions between them, while the weights represent the probabilities or frequencies of the state and transition occurrences. The same applies to multiple behaviours sharing the same set of possible states. In analysis of movement data, state transition graphs can be used to represent semantic abstractions of mobility behaviours, where states correspond to semantic categories of visited places (such as ‘home’, ‘work’, ‘shop’), activities of moving objects (‘driving’, ‘walking’, ‘exercising’, etc.) or characteristics of the movement (‘straight movement’, ‘sharp turn’, ‘acceleration’, ‘stop’, etc.). Such a representation supports the exploration and analysis of the semantic aspect (i.e. the meaning or purposes) of movement. For comprehensive analysis of movement data, state transition graphs need to be combined with representations reflecting the spatial and temporal aspects of the movement. This requires appropriate coordination between different visual displays (graphs, maps and temporal views) and appropriate reaction to analytical operations applied to any of the representations of the same data. We define in an abstract way the reactions of a graph display to analytical operations of querying, partitioning and direct selection. We also propose visual and interactive display features supporting comparisons between data subsets and between results of different operations. We demonstrate the use of the display features by examples of real-world and synthetic data sets. 

State Transition Graphs for Semantic Analysis of M

State Transition Graphs for Semantic Analysis of Movement Behaviours

Abstract Numeric time series is a class of data consisting of chronologically ordered observations represented by numeric values. Much of the data in various domains, such as financial, medical and scientific, are represented in the form of time series. To cope with the increasing sizes of datasets, numerous approaches for abstracting large temporal data are developed in the area of data mining. Many of them proved to be useful for time series visualization. However, despite the existence of numerous surveys on time series mining and visualization, there is no comprehensive classification of the existing methods based on the needs of visualization designers. We propose a classification framework that defines essential criteria for selecting an abstraction method with an eye to subsequent visualization and support of users' analysis tasks. We show that approaches developed in the data mining field are capable of creating representations that are useful for visualizing time series data. We evaluate these methods in terms of the defined criteria and provide a summary table that can be easily used for selecting suitable abstraction methods depending on data properties, desirable form of representation, behaviour features to be studied, required accuracy and level of detail, and the necessity of efficient search and querying. We also indicate directions for possible extension of the proposed classification framework.

Data Abstraction for Visualizing Large Time Series

Analysis of action sequence data provides new opportunities to understand and model user behaviour. Such data are often in the form of timestamped and labelled series of atomic user actions. Cyber security is one of the domains that show the value of the analysis of these data. Elaborate and specialised models of user-behaviour are desired for effective decision making during investigation of cyber threats. However, due to their complex nature, activity sequences are not yet well-exploited within cyber security systems. In this paper, we describe the initial phases of a visual analytics approach that aims to enable a rich understanding of user behaviour through the analysis of user activity sequences. First, we discuss a motivating case study and discuss a number of high level requirements as derived from a series of workshops within an ongoing research project. We then present the components of a visual analytics approach that constitutes a novel combination of “action space” analysis, pattern mining, and the interactive visual analysis of multiple sequences to take the initial steps towards a comprehensive understanding of user behaviour.

A Visual Analytics Approach for User Behaviour Und

A Visual Analytics Approach for User Behaviour Understanding through Action Sequence Analysis

The correlated exploitation of heterogeneous data sources offering very large historical as well as streaming data is important to increasing the accuracy of computations when analysing and predicting future states of moving entities. This is particularly critical in the maritime domain, where online tracking, early recognition of events, and real-time forecast of anticipated trajectories of vessels are crucial to safety and operations at sea. The objective of this paper is to review current research challenges and trends tied to the integration, management, analysis, and visualization of objects moving at sea as well as a few suggestions for a successful development of maritime forecasting and decision-support systems.

Maritime data integration and analysis: recent pro

Maritime data integration and analysis: recent progress and research challenges

A key problem in the field of search interfaces is dyslexic users interaction with the UI. Dyslexia is a widespread specific learning difficult (SpLD) (10% of any population is estimated to have this cognitive disability) which is under researched in the field of information retrieval. The focus here is an analysis of the User Interface (UI) for search, using visual analytical methods on eye tracking data to examine the difference between control and dyslexic searchers. We use a number of visual analytic methods including path similarity analysis (PSA) and clustering of time intervals to demonstrate both similarities and differences between the user groups. Observations of videos are used to augment the visualizations. Results demonstrate a clear difference between the user groups, and a clear memory effect on the user of search interfaces is shown -- this is a key contribution of this paper. We examine the results using of theories of dyslexia, contributing also to the field of dyslexia and search.

Visual Analysis of Dyslexia on Search

The visual analysis of eye movement data has become an emerging field of research leading to many new visualization techniques in recent years. These techniques provide insight beyond what is facilitated by traditional attention maps and gaze plots, providing important means to support statistical analysis and hypothesis building. There is no single ``all-in-one'' visualization to solve all possible analysis tasks. In fact, the appropriate choice of a visualization technique depends on the type of data and analysis task. We provide a taxonomy of analysis tasks that is derived from literature research of visualization techniques and embedded in our pipeline model of eye-tracking visualization. Our task taxonomy is linked to references to representative visualization techniques and, therefore, it is a basis for choosing appropriate methods of visual analysis. We also elaborate on how far statistical analysis with eye-tracking metrics can be enriched by suitable visualization and visual analytics visual analyticstechniques to improve the extraction of knowledge during the analysis process.

A Task-Based View on the Visual Analysis of Eye-Tr

A Task-Based View on the Visual Analysis of Eye-Tracking Data

Modern movement tracking technologies enable acquisition of high quality data about movements of the players and the ball in the course of a football match. However, there is a big difference between the raw data and the insights into team behaviors that analysts would like to gain. To enable such insights, it is necessary first to establish relationships between the concepts characterizing behaviors and what can be extracted from data. This task is challenging since the concepts are not strictly defined. We propose a computational approach to detecting and quantifying the relationships of pressure emerging during a game. Pressure is exerted by defending players upon the ball and the opponents. Pressing behavior of a team consists of multiple instances of pressure exerted by the team members. The extracted pressure relationships can be analyzed in detailed and summarized forms with the use of static and dynamic visualizations and interactive query tools. To support examination of team tactics in different situations, we have designed and implemented a novel interactive visual tool ``time mask''. It enables selection of multiple disjoint time intervals in which given conditions are fulfilled. Thus, it is possible to select game situations according to ball possession, ball distance to the goal, time that has passed since the last ball possession change or remaining time before the next change, density of players' positions, or various other conditions. In response to a query, the analyst receives visual and statistical summaries of the set of selected situations and can thus perform joint analysis of these situations. We give examples of applying the proposed combination of computational, visual, and interactive techniques to real data from games in the German Bundesliga, where the teams actively used pressing in their defense tactics.

VA of pressure in football

Visual analysis of pressure in football

Many cities and countries are now striving to create intelligent transportation systems that utilize the current abundance of multisource and multiform data related to the functionality and the use of transportation infrastructure to better support human mobility, interests, and lifestyles. Such intelligent transportation systems aim to provide novel services that can enable transportation consumers and managers to be better informed and make safer and more efficient use of the infrastructure. However, the transportation domain is characterized by both complex data and complex problems, which calls for visual analytics approaches. The science of visual analytics is continuing to develop principles, methods, and tools to enable synergistic work between humans and computers through interactive visual interfaces. Such interfaces support the unique capabilities of humans (such as the flexible application of prior knowledge and experiences, creative thinking, and insight) and couple these abilities with machines' computational strengths, enabling the generation of new knowledge from large and complex data. In this paper, we describe recent developments in visual analytics that are related to the study of movement and transportation systems and discuss how visual analytics can enable and improve the intelligent transportation systems of the future. We provide a survey of literature from the visual analytics domain and organize the survey with respect to the different types of transportation data, movement and its relationship to infrastructure and behavior, and modeling and planning. We conclude with lessons learned and future directions, including social transportation, recommender systems, and policy implications.

VA of Mobility and Transportation: State of the Ar

Visual Analytics of Mobility and Transportation: State of the Art and Further Research Directions

We introduce the concept of time mask, which is a type of temporal filter suitable for selection of multiple disjoint time intervals in which some query conditions fulfil. Such a filter can be applied to time-referenced objects, such as events and trajectories, for selecting those objects or segments of trajectories that fit in one of the selected time intervals. The selected subsets of objects or segments are dynamically summarized in various ways, and the summaries are represented visually on maps and/or other displays to enable exploration. The time mask filtering can be especially helpful in analysis of disparate data (e.g.,event records, positions of moving objects, and time series of measurements), which may come from different sources. To detect relationships between such data, the analyst may set query conditions on the basis of one dataset and investigate the subsets of objects and values in the other datasets that co-occurred in time with these conditions. We describe the desired features of an interactive tool for time mask filtering and present a possible implementation of such a tool. By example of analysing two real world data collections related to aviation and maritime traffic, we show the way of using time masks in combination with other types of filters and demonstrate the utility of the time mask filtering.

interactive time masks

Visual exploration of movement and event data with interactive time masks

This paper is concerned with the representation and exploration of flows, defined as spatial interactions between geographic locations. Flows are challenging to display in a comprehensible manner due to the nature of the data, which are characterized by many crossings and overlaps leading to clutter. A number of different strategies have been suggested for addressing this problem, which commonly involve reducing the search space, aggregating the data or simplifying the representations often at the cost of information loss or distortion of spatial context. We propose an interactive approach for exploring large and highly connected networks of flows without distorting the geographical space and without losing the context overview in the process. The approach is based on a flow-specific interaction technique for filtering the data by direction, that enables an analyst to successively identify underlying spatial arrangement patterns. We illustrate our approach through exploring flows of tourists and locals in the Greater London area.

Exploration of Flows Through Direction-Based Filte

An Interactive Approach for Exploration of Flows Through Direction-Based Filtering

Trajectory-based visualization of coordinated movement data within a bounded area, such as player and ball movement within a soccer pitch, can easily result in visual crossings, overplotting, and clutter. Trajectory abstraction can help to cope with these issues, but it is a challenging problem to select the right level of abstraction (LoA) for a given data set and analysis task. We present a novel dynamic approach that combines trajectory simplification and clustering techniques with the goal to support interpretation and understanding of movement patterns. Our technique provides smooth transitions between different abstraction types that can be computed dynamically and on-the-fly. This enables the analyst to effectively navigate and explore the space of possible abstractions in large trajectory data sets. Additionally, we provide a proof of concept for supporting the analyst in determining the LoA semi-automatically with a recommender system. Our approach is illustrated and evaluated by case studies, quantitative measures, and expert feedback. We further demonstrate that it allows analysts to solve a variety of analysis tasks in the domain of soccer.

Dynamic Visual Abstraction of Soccer Movement

Origin-destination (OD) movement data describe moves or trips between spatial locations by specifying the origins, destinations, start, and end times, but not the routes travelled. For studying the spatio-temporal patterns and trends of mass mobility, individual OD moves of many people are aggregated into flows (collective moves) by time intervals. Time-variant flow data pose two difficult challenges for visualization and analysis. First, flows may connect arbitrary locations (not only neighbors), thus making a graph with numerous edge intersections, which is hard to visualize in a comprehensible way. Even a single spatial situation consisting of flows in one time step is hard to explore. The second challenge is the need to analyze long time series consisting of numerous spatial situations. We present an approach facilitating exploration of long-term flow data by means of spatial and temporal abstraction. It involves a special way of data aggregation, which allows representing spatial situations by diagram maps instead of flow maps, thus reducing the intersections and occlusions pertaining to flow maps. The aggregated data are used for clustering of time intervals by similarity of the spatial situations. Temporal and spatial displays of the clustering results facilitate the discovery of periodic patterns and longer-term trends in the mass mobility behavior.

Revealing Patterns and Trends of Mass Mobility Thr

Revealing Patterns and Trends of Mass Mobility Through Spatial and Temporal Abstraction of Origin-Destination Movement Data

 ABSTRACTOver four workshops, we collected community input on what people considered as persistent challenges in geovisualization with the participation of 72 experts from various sub-domains of geographic information science and technology. We categorize and analyze this bottom-up input, and contrast it with the previously published research challenges based on five research agenda papers (top-down). We observe certain overlaps but also some interesting differences between the top-down and bottom-up approaches. A synthesis of the two suggests three major issues as persistent challenges: (1) a better understanding of the scope of our domain, how it interacts with other domains, and how to make this happen, (2) a systematic understanding of human factors, (3) a ‘practicable’ set of guidelines that matches the visualization types to task types, and guides the practitioner to design geovisualizations that are appropriate and helpful to the user. Distinguishing persistent from important, we discuss why the identified challenges are persistent, and draw recommendations for action based on our observations and interpretations. We believe these findings will help building a stronger, better-grounded research agenda for our community. 

Persistent challenges in geovisualization – a comm

Persistent challenges in geovisualization – a community perspective

 AbstractUnderstanding of data quality is essential for choosing suitable analysis methods and interpreting their results. Investigation of quality of movement data, due to their spatio-temporal nature, requires consideration from multiple perspectives at different scales. We review the key properties of movement data and, on their basis, create a typology of possible data quality problems and suggest approaches to identify these types of problems. 

Understanding movement data quality

 ABSTRACTThe problem we address is prediction of expected values of some attribute of spatial objects based on values of other attributes, including the geographic positions. A common approach to obtaining such predictions is regression modelling. It is highly desirable that predictive models are not only accurate but also understandable to the users, which gives preference to simpler models. We propose a set of visualisation techniques and interactive operations that supports exploration, evaluation, refinement and simplification of regression tree models. In particular, the analyst can investigate how the model components and their properties are related to the spatial distribution of the objects, and can make the model better account for the spatial aspect of the data by generating new space-based attributes and supplying them to the model building tool. 

regression tree models

Exploration and refinement of regression tree models with interactive maps and spatial data transformations

Learning more about people mobility is an important task for official decision makers and urban planners. Mobility data sets characterize the variation of the presence of people in different places over time as well as movements (or flows) of people between the places. The analysis of mobility data is challenging due to the need to analyze and compare spatial situations (i.e., presence and flows of people at certain time moments) and to gain an understanding of the spatio-temporal changes (variations of situations over time). Traditional flow visualizations usually fail due to massive clutter. Modern approaches offer limited support for investigating the complex variation of the movements over longer time periods. We propose a visual analytics methodology that solves these issues by combined spatial and temporal simplifications. We have developed a graph-based method, called MobilityGraphs, which reveals movement patterns that were occluded in flow maps. Our method enables the visual representation of the spatio-temporal variation of movements for long time series of spatial situations originally containing a large number of intersecting flows. The interactive system supports data exploration from various perspectives and at various levels of detail by interactive setting of clustering parameters. The feasibility our approach was tested on aggregated mobility data derived from a set of geolocated Twitter posts within the Greater London city area and mobile phone call data records in Abidjan, Ivory Coast. We could show that MobilityGraphs support the identification of regular daily and weekly movement patterns of resident population.