The Data Lenses is a visual exploration tool for several attributes of the same dataset: buses in Singapore. As described in the video, it enables to select various types of data, while speeding up and slowing down the simulation. The data that it reads is not pre-processed and therefore is aggregated on the fly on a bus level.

Nevertheless, what interests me more here is the experiment that his tool is on visualization per se. I wanted to provide a great amplitude of zoom levels without the classical pan and zoom that often gets me lost. The classical solution for this is the fish-eye lens. The problem is that the typical fish-eye does not carry a zoom level as great as this one: from the island overview to the narrow contemplation of the street. Other solutions can pass by just mapping a magnified circle over the interest point but this obviously brings the occlusion of the periphery of the magnified location, destroying the experience of surroundings’ orientated browsing.

After trying to distort the space around a point in all sorts of ways, I came up with a distortion strategy that implements a lens equation of a somehow surreal nature. A point is distorted in function of its current radius to the center of the lens. This distortion rate varies with an arctangent and a square root (after trying all sorts of combinations of gaussians and quadratic functions). In the end what I was looking for was a function with a nice sexy shape, that starts in zero, escalates to a maximum and decreases with an almost gaussian shape and then quickly approaches zero like an hyperbolic. My quest resulted in the following function that expresses the distorted radius to a lens center (instead of the non-distortion in blue: y=x).

This implementation results in a much more diverting and interesting browsing of the space, where I can direct the lens to a clutter of points, and fluidly unveil and understand each of its constituting parts.

A purely technical aspect of this tool is that even the map is not pixel based, but node based, meaning that every distortion is applied to a vertice and not a pixel coordinate. This enables to apply high magnification levels without getting any pixelation on the map. Nevertheless, since we are dealing with large quantities of vertices, the map is pushed to the graphic card’s memory using Vertex Buffered Objects and the distortions are computed on the GPU’s Vertex Shader, described with GLSL and using GLGraphics.

Acknowledgements

This tool was developed in the framework of visual explorations of urban mobility of LIVE Singapore! project. Special thanks go to Kristian Kloeckl for the supervision and trust, Till Nagel and Inês Dias for the untangling suggestions, Nicholas Marchesi and Prudence Robinson for baring with headaches while doing the showcase video.

Usually, in my projects, I am most interested in coming up with the visual outcomes of certain approaches, accounting if the approach itself is of relevant interest, intellectual honesty and conceptual elegance. For the Color Forecast I developed the color clustering software, being framed within a straightforward concept (without known applications in the fashion context), delivered on a beautifully designed website, iPhone app, and presented with joyful short videos. Having a project that results in something so roundly defined and with an aesthetic tone that I’m found of in this context, it is a great pleasure to contribute with the low-level intricacies, that in spite of being under the hood, make it work.

The process is broadly described in the following video. Albeit the effort of Happiness Brussels in publicizing the existence of these technical intricacies, or how it is called, technology, I feel the necessity of describing my contribution, explaining the devoted effort and clarifying the overall simplicity in the approach.

Frame differencing

The cameras are meant to be installed pointing at pedestrian streets, where generally the only things that move are people and no vehicles are expected. To get rid of the backgrounds that are of no importance in detecting the colors that people are wearing, the application only captures moving pixels through frame differencing. These moving pixels are stored during fixed periods of time (for example, 3 minutes, after when the pixels are discarded and a new set of temporal moving pixels starts being recorded again).

3D histogram

Since the application has to be running on 24/7, special attention had to be taken on the data structures used, in order to assure performance and avoid memory leaks. The temporal histogram, for example, is stored on a 255 × 256 × 256 matrix, so instead of storing a pixel, which coordinates are of no interest in this simple approach, only a specific entrance of the matrix is incremented in order count the number of occurrences of a certain color which RGB values [0-255] are expressed in the matrix coordinates.

K-means

Each time the capture period is over, a K-means clustering runs over a copy of the histogram on a separate thread. Typically the number of clusters ranges from 10 to 20, and can be setup together with other important variables (camera resolution, framerate, threshold for frame differencing, accumulation period, name of the city and database scripts). It is important to notice that this K-means clustering is only interested in grouping colors, therefore not accounting for the amount of occurrences of a color, but only if it occurred or not.

Seeds

Since this K-means is meant for Pimkie, the seeds for the clusters are fixated at white, black, and all the equally distant pure hues that are left for the K-2 clusters.

Color distance

The K-means needs a distance measure between colors. I tried several approaches for this, from a RGB Manhattan distance to improve performance, to an HSB distance and passing through a perceptual euclidean distance measure ( √(0.26R + 0.70G + 0.04B), referred in Gijenij et al. “A Perceptual Comparison of Distance Measures for Color Constancy Algorithms“, ECCV 2008). After several experiments, the distance measure that presented the best clustering results, from of course my empirical subjective observation, was just the straight forward euclidean distance between R, G and B.

Averaging colors

The K-means algorithm also requires a procedure to compute the mean color of a cluster, iteratively adapting the cluster’s centroid during the K-means routine. Again, the approach that presented the best results for this was just the average of R, G, B.

Representative color

After having the colors clustered in K groups, it is time to compute the representative color for each cluster. This mean color is not just the simple mean of colors, but a weighted average of colors by their number of occurrences (because if in a group of reds, there are much more pinks than maroons, then the representative color should be more pink than maroon).

Color perception

Now that we have these K groups of colors, each with a specified amount of occurrences, how to say that one group is more important, or perceptible than another? After-all we are interested in finding the most trending color of the moment. In my point of view, and without academic formalisms, if everybody is grey, dull or dark, and a magnificent woman parades with a yellow scarf, I will say that the most perceptible color in the scenery is yellow. So, after running an image segmentation algorithm (a simple region growing one) over a database of about one thousand images, the empirical measure that I came up with to describe how vibrant a color can be is: (saturation × brightness)² × number_of_occurrences. So the color clusters are sorted not only by how many occurrences of moving pixels they represent, but also weighted by a quadratic importance of saturation and brightness of their representative color.

Normalization

After each K-means concluded, its results are submitted to a database, in the format of: K colors sorted by order of importance; the corresponding absolute number of occurrences; and the normalized (by maximum) number of occurrences, because color perception is dependent of the temporal accumulation context and should not be assumed that the yellow scarf woman that passed today in front of the camera (more yellow moving pixels) is less or more important than that same yellow scarf woman that passed yesterday farther from the camera.

Ongoing project

For now, colors are being clustered for Milan, Antwerp and Paris and might extend to other cities. Feedback is being absorbed in order to improve the computational approach accuracy and corresponding representation of the most trendy colors on specific streets.

Celebrating colors

And as this project is all about colors, I share a very old visual essay that I did about the nature and harmony of colors (in CMYK, though).

This joint adventure with graphic design studio FBA. has been sitting in the closet for more than a year, and recently started bearing fruits. It resulted in a tool that helps designers build visual artifacts to use in book covers design. The tool parses a text and exhibits its most frequents words, enabling the designer to choose which words he/she wants to visualize by also choosing from more than a dozen of visualization models. Despite the recurring use of the word “visualization”, the intent here is not to visualize anything, but to provide abstract visual artifacts that are innately related with the form of the text or content of the book (the semantic capabilities of the tool are meager, but this doesn’t diminish what I believe is a proper strategy: if I want to design something about a text, then I can start by analytically giving form to that text).

The tool can map occurrences of selected words across the whole text. For instance, taking Tolstoy’s War and Peace, we can map Pierre Bezukhov, Natasha Rostov, Andrei Bolkonsky (in shades of blue) and Napoleon Bonaparte, Tsar Alexander I (in shades of red). As a result we have several visual artifacts from different visualization models. Each model has its own varying parameters that can create drastically different results. Following, I excuse myself from the boredom of describing each visualization model, inputs and parameters, but they pass through dotted maps, temperature maps, circular and bar graphs, texture based density maps (a la 60′s statistical maps) and even particles trajectories that are attracted to the occurrences of certain words. Andrew Vande Moere suggested that describing the visualization models could be of use to the visualization community, and indeed he is very right since most of the visualization showcase articles do not describe properly the models in use. When I started reading about interesting visualization projects some years ago, the absence of cut to the chase descriptions was a harsh obstacle in the learning process. Therefore, some of the visualization models are briefly described in the end of this article.

Taking these artifacts to the purpose of the tool (book covers design), I made some experiences based on Ana Boavida/FBA.’s designs for Minotauro collection of latin-american writers. For instance, I tend to select the characters of the book, coloring only the protagonists.

Finally, realizing that this could work, we waited for projects in which this strategy could make perfect sense. The first real results are a growing collection about management, where the book covers were designed by Rita Marquito/FBA. using this tool. The whole management thematic is abstract enough to scream for this kind of approach.

What is interesting about this tool is that it is flexible enough to allow the designer to develop different forms of expression. And how Ana Boavida/FBA. did this… For the next collection of law books, she parameterized a visualization model with combinations that I hadn’t predicted, thus creating regular patterns that in spite of giving great book covers, expunge the data nature of the artifacts. But that, is another story.

Photographs by Daniel Santos/FBA.

Brief description of visualization models

The visualization models pass through dotted maps, temperature maps, circular and bar graphs, texture based density maps (a la 60′s statistical maps) and even particles trajectories that are attracted to the occurrences of certain words.

A text in a book is typically organized in semantic aggregations (e.g. chapters, sections). Since the tool does not try to extract any semantic information by itself, I start by considering the text as one big line of words. The trick here is to pass this unidimensional nature to a 2D canvas. Histograms do it, but I prefer starting with maps. Therefore most of the models start by lying out the text through several horizontal lines, as if one page could represent the whole book.
This way one can easily map in which portions of the text a certain character appears.

With this layout the position of each occurrence is determined. A seen on the left, I can represent each occurrence as a square. The color of the square is the attributed color to the corresponding word, while the size of is proportional to the total of occurrences in the text. Less occurring words are drawn above most frequent words in order to avoid occlusions.

Instead of drawing squares over the occurrences, the colors of the pixels can be directly set weighting the proximity to word occurrences. The strategy adopted on the visualization at left only accounts for occurrences that are in the past, generating abrupt changes in color when unexpected occurrences are encountered while getting a washed out effect as the strength of each occurrence decreases while reading advances.

The text can also be divided in N portions of equal size. In each of these portions, if there are occurrences of a certain word, a corresponding pattern is drawn. The patterns for each selected word are dynamically generated through the rotation of a basic line-based pattern, assuring that each word has an unique pattern. In this way each portion of the text is represented trough an artifact that can be used do decipher the density of important occurrences and the similarity with other portions.

Perhaps the most adventurous visualization in the 2D realm is depicted at left. There is one trajectory for each selected word through its occurrences. The initial position is determined by the strategy previously described. The trajectory is then incremented as if the text was being advanced, curving as the reading position approximates a new occurrence. The qualitative dimension of the fur balls traduces an agglomeration of occurrences, so if an occurrence is isolated, the trajectory quickly converges to a point without much spinning. When the next occurrence is far ahead on the text, the trajectory will be rectilinear. Because I haven’t found the mixture of rects and curves of aesthetic interest, the rectilinear trajectories can be removed. When a trajectory spans canvas’ bounds it emerges in the opposite edge as like in a toroidal space. The intensity of the curvature and the looking ahead distance that aggregates the next occurrences are customizable, being able to generate a great variety of artifacts from the same visualization model.

The next models aren’t based on the previously described positioning strategy. Instead they revolve in turn of the unidimensional nature of the text. At left, an atomic histogram of occurrences, where the height is the total number of occurrences of the corresponding word class.

The same histogram but with a circular base.

Finally, this last visualization model fills a shape with vertexes based in the circular histogram. Therefore, the disruption of each circular shape represents the absence of the corresponding word in portions of the text.

In this work the traffic of Lisbon is portrayed exploring metaphors of living organisms with circulatory problems. Rather than being an aesthetic essay or a set of decorative artifacts, my approach focuses on synthesizing and conveying meaning through data portrayal. This portrayal is embodied in the visualization: The Blood Vessels in the traffic of Lisbon. I use an adaptive physics system to build and manipulate the road network – the thickness, the color and the length of the vessels are excited by the number of vehicles and average velocity in each road. With this system I try to bypass the strictness of contemporary visualizations that depict data accurately through direct mappings.

The road network of Lisbon was queried from OpenSreetMap, parsed and filtered. Using this information, a spring based physics system is build for the road network and a filling structure of each vessel. The data is overlaid on the resultant structure to determine the road where each vehicle is at a given moment. This allows to inject data at runtime and excite the system as follows: a greater number of vehicles on a vessel tend to make it thicker, higher speeds tend to contract its length (an vice-versa). The latter behavior was chosen in order to transmit a global impression of the perceived distances within the city. This behavior shrinks the city when the traffic velocities are higher, and distends it in the rush hours when the city faces congestion problems. In what concerns coloring, lower speeds imply the darkening of a vessel, expressing slower circulation and stagnant blood.

The vessels’ visualization, with crude aesthetics that are innate to the visual metaphor of an organism with circulatory problems, pulsates for each rush hour and stresses precisely which roads are congested. For example, in the following figure the most problematic roads are (without order):

1. CRIL
2. Segunda Circular
3. Eixo Norte-Sul
4. CREL
5. A5
6. Marginal / EN6

As a matter of curiosity (or not), here are some previous failed attempts to attain this metaphor using swarms and particles — I call them the hairy roads :p

The Morphing City is a visualization study where a city mutates its shape accordingly with the traffic on its main arteries. Those morphs tend to traduce the actual perceived distances within a city, bypassing the common perception based on its geographical mapping.

This visualization model was executed for the city of Lisbon. To attain it, topological information was gathered from OpenStreetMap to build a skeleton for the city based on its main arteries. The bones of the skeleton are springs that get compressed or distended accordingly with the detected velocities over it. Those distortions affect all the neighboring points and springs as the system is all interconnected.

The data concerning the velocities was gathered in the context of the CityMotion project, and it pays respect to 1534 vehicles in the city of Lisbon during October 2009. That data was averaged to a single day, and aggregated by periods of one hour. Those periods overlap in 50min, meaning that they are iterated by ticks of 10min.

What is being displayed are the distortions on each artery that affect the entire city. If the current speed on that artery is below its average global speed, the artery is compressed (the higher the velocity, the smaller the perceived distance). Similarly, if the speed is over the computed (during pre processing) global average, the artery is distended. The colors also reflect those distortions, with positive deviations traducing warm colors, and negative deviations traducing cold colors.

Another way to perceive the morphs in the city during the day it’s the deformed grid on the bottom right corner of the video (and below — distorted grid at 17h).

It’s interesting to notice how the city stays compressed during the evening, and how it abruptly expands during the rush hours: 8h-9h and 18h-19h. It’s also interesting to see that the 8h-9h period is by far the most problematic.

And that’s the system behind those artifacts. Believe it or not, the system reacts to the data concerning the traffic in Lisbon during October 2009 from 17h to 18h.

What’s behing the fudge from Pedro M Cruz on Vimeo.

Too bad that my pseudo-scientific consciousness doesn’t allow me to start working on an arty concept that could justify the mere existence of this artifact. It exists, for me, because in the end it’s a big LOL.

Glitching Lisbon from Pedro M Cruz on Vimeo.

There is more information displayed, as the former colonies persist on the map and head to their current geographical positions. Therefore it is possible to visualize in the end of the narrative how much of the world was once part of an Empire. The timeline is no longer linear as it speeds up if there is nothing going on.

The music was kindly composed for the purpose of this narrative by CHOP WOOD – chopwood.eu

Traffic clots

Perhaps the strongest visual metaphor — an organism with circulatory problems. When the traffic is slow, the red clots start appearing. Watch video on Vimeo.

Aesthetic deviation

The following artifact is in its essence an aesthetic deviation that brings a visual emphasis on the main traffic routes. Pencil and paper. Darker arteries traduce intense traffic. Watch video on Vimeo.

Searching for emphasis in sluggish areas

In the next artifact the color of the arteries change with the speed of the vehicles. Rapid transit arteries are drawn with greenish and cooler colors, while the sluggish ones are reddish and hotter. Traffic intensity is mapped in the thickness and brightness of the arteries. When a certain path has a very low velocity, its covered area gets filled — the intention was to bring an emphasis to the areas with major traffic issues (low speed). The white dots represent the vehicles themselves. Eye candy, eye candy. Watch video on Vimeo.

These visualization were developed in the context of CityMotion and my master thesis at DEI/FBA.

SyncLost is a multi-user installation for immersion in the history of electronic music. From a complex timeline, rhythms and sub-rhythms merge to create new sounds.

The project’s objective is to create an interface where users can view all the connections between the main styles of electronic music through visual and audible feedback. The choice is individual and leads to a collective consequence in the spatial visualization of information.

Give it a look! More on the project here.