Squatting Supermarkets.

iSee and the emergence of the ethical social consumer.

Published in: Proceedings of the 3rd Universal Design Conference
Hamamatsu City, Japan, 30 October - 3 November 2010.
Federico Ruberti, Salvatore Jaconesi, Luca Simeone.

The most recent developments in technology and mobile communications, along with the new relational P2P models and the relevance of environmental sustainability, suggest that in the following years we will see a scenario that radically differs from the current situation within the convenience goods industry. The traditional dynamics of shopping characterised by the model of “one - all”, from businesses to consumers, already seem inadequate for a significant part of the population; one can hypothesise that in the near future this model would be substituted by practises based on narrative, hyper textual, multimedia and distributed models. Smart tagging, augmented reality, user generated content, ubiquitous publishing etc, may already form the framework of a new paradigm of shopping in which businesses and consumers establish a necessarily transparent dialogue on the actions taken in the construction of a sustainable future. Squatting Supermarkets prefigures, outlines and fulfils the model of the shopping narrative described above. From both technological and socio-economic points of view they confront the topic of the exposure and dispersion of layers of information that are useful in the practice of buying, through an immersive ambient that is interactive and stimulating to the senses. Squatting Supermarkets are able to outline a dialogue in which the histories, processes, the value chain, and the consumer himself become visible and accessible in a narrative and socially relevant form.

Social Consumerism; Social Design; Augmented Reality; Ubiquitous Publishing; Corporate Social Responsibility.

Introduction and problem statement.
In the culture of experience, the value of merchandise is tied to the relationship it establishes with the consumer [Pine, J. II; Gilmore, J: 1999], it is for this reason that most shopping malls appear to us as theatres that intoxicate our senses and put on stage long expanses of packets that are designed to give clues and cross-references with their imagery; the supermarkets present themselves as theatres of consumption, a stage on which merchandise turns into spectacles through the use of display stands, olfactory stimulants and interactive monitors for consumers who are committed to products that best represent their desired experiences. Yet, in an attempt to look at tomorrow’s consumer through analysing today’s phenomena, the communication strategies that are put into use in the shops by the production and distribution companies seem destined to concede to profoundly different dynamics. These are caused by phenomena set in motion by the development of a sensibility towards the topic of environmental sustainability, the importance assumed by the User Generated Content and the ubiquitous accessibility of technology.

It is on this aspect, and on its socio-economic implications, that Squatting Supermarkets dwell. This project was developed by FakePress with the aim of prefiguring, delineating and putting into practice a model for shopping in which layers of socially relevant information superimpose products, stabilising a transparent relationship between businesses and consumers directly at the shop floor.

The emerging of the ethical social consumer

Corporate Social Responsibility has become a determining factor in the creation and maintenance of a brand’s good reputation and the consequent appreciation of its products on the market. The relevance of this phenomenon to marketing is demonstrated in the conferences and the research that concentrate on the commercial implications that a good strategy of communication and sustainable practices could have. According to recent studies [1] 80% of consumers at an international level are sensitive to the environmental issue; despite global recession, 48% of consumers are prepared to pay 10% more for goods made with socially responsible and eco-friendly means. The shop becomes the place of fulfilment of an often manipulative strategy that exploits the suggestions of sustainability in order to attract favour:  J. Celia [2], remarks that marketing represents the link between the commercial world and the consumers, a link which ever more frequently manifests itself at the moment of their decision of acquisition with the aim to modify behaviour and choices. He goes on to say that it is for this reason that we cannot refrain from asking ourselves what our role is in the specific area of choice of acquisition and sustainable consumption.

It is sufficient to analyse recent Mintel [3] data related to packaging and the key messages of communication used for launching new products at a global level to see how in many sectors the key words used belong mainly to the fields of health and environment; biological, natural, without preservatives, no added chemicals, natural ingredients, low environmental impact etc. Ethical consumerism has evolved in recent years and today it sees consumers, not only as being individually orientated towards awarding virtuous companies based on the “suggestions of traditional communication”, but also as being active in research and in sharing information, with an intention to influence the definition of a corporate offer with their own decisions of acquisition and consumption. [

De Luca, P.: 2006]. The packaging and the communication strategy, from the viewpoint of the emerging consumer for tomorrow’s shopping, does not seem sufficient in as much as it doesn’t take into consideration the social control that is presently developing through spontaneous and self-organising dynamics. In an internet search carried out on 20 of the world’s most famous brands 25% of the results were links to User Generated Content; on the web 34% of bloggers express comments and opinions on products and brands to a public of whom 78% trust their recommendations [Qualman, E.: 2009].

These factors, together with the development of ubiquitous technology, require that production and distributing companies consider the new dynamics of the creation of acquisition preferences. The groundswell, that is, the trend of obtaining information and goods from other individuals rather than from businesses and institutions, assumes, along with the evolution of mobile technology, a disruptive potential which companies must face. “As powerful as it is, technology is just an enabler. It’s the technology in the hands of almost always connected people that makes it so powerful.” [Li,C.; Bernoff, J.:2008, p.4]. According to the Pew Internet and American Life Project [Purcell, K.; Rainie, L.; Mitchell, A.; Rosenstiel, T.; Olmstead, K.: 2010)], people’s relationship to information is increasingly becoming portable, personalized, and participatory: 33% of mobile phones owners access news on their phone, 28% of Internet users customise their home page with news from third party sources 37% of Internet users contribute to the creation of news, comment about it, or disseminate it via blogs and social media web sites.

Ubiquitous technologies represent an extremely qualifying factor towards which global demand doesn’t seem to be slowing, even in times of recession. The market for smart phones is one of the few sectors of growth, so much so that in the first semester they exceeded 36.4 million units, with a growth of 12.7% [4] in respect to the previous year. According to “Key Predictions for IT Organizations and Users in 2010 and Beyond” by Gartner [5], by 2014 the quota for penetration of mobile telephony will reach 90%, with 6.5 billion mobile connections. The number of PCs in use will reach 1.8 billion units by 2013, whilst the total of smart phones and mobile phones with access to the internet will exceed this. The recently defined “social consumer” [Michetti, N., Cosenza, V. 2009] is a consumer who is always online, connected via various devices, (computer, mobile phones etc.) and who is, above all, proactive: participating through his own blog, tracking down information by consulting many different sources and spending much time on social media and the blogs of others. This consumer is more trusting of his peers than of institutions and companies, and he is active in his choices, he believes in sharing information.

The phenomena described here suggest that there is an emerging type of consumer who will question reports given by production and distribution companies: a consumer who is attentive to eco-sustainable practices, who is always connected to the web via mobile devices, who is prepared to share his own knowledge and impressions through collaborative practices, in other words, an “ethical social consumer”. It is towards this type of consumer that Squatting Supermarkets and its Mobile application, iSee, are directed.

iSee is a mobile application conceived, designed and developed by FakePress, a next-step publishing house leveraging SPIMEs, location based technologies, distributed interactions and emotional and experience design, with the aim to match the social and cultural phenomena mentioned above and to fulfil a kind of shopping in which consumers may access any product’s information related to corporate social responsibility, environment and sustainability; information that is provided by companies, independent analysts and users through their mobile phone cameras.

See application and Squatting Supermarkets

The first version of the iSee application, developed in autumn 2009, recognises logos of the most widespread detergents on the European market. The application is composed of a visual pattern recognition engine, connected to a database, which contains the visual files of the logos, the pre-processed meta-data needed to recognize them and the related information and educational content. A CMS (content management system) allows for the loading of new logos, training the pattern-recognition system in recognizing them, and associating information and content with these logos.

The CMS is also connected to a social media platform that allows end users to upload their comments or reviews and to associate them to a specific logo. People using the application/publication can thus turn logos into discussion spaces as communication infrastructures on which to promote organic products as alternatives to industrially-produced ones, or as opportunities to create dialogues and information flows dealing with ecology, sustainability and alternative economies and to discuss critical, day-to-day practices.

Fig. 1 iSee Application. The phone’s built-in camera takes a picture of the detergent pack, the pattern recognizing system detects the producer’s logo and the application retrieves information about the product and its producer.

The application retrieves information and educational content from several sources:

  • Websites such as Corpwatch.com and CorporateWatch.org that track corporate responsibility policies and ethical issues of big corporations
  • SPIMEs (distributed sensors) that monitor in real-time the level of pollution related to detergents in the sea nearer to the user, in particular the level of POP (Persistent organic pollutants) and eutrophication coming from surfactants. This real-time information comes from data collected on NASA Global Change Master Directory
  • A selection of educational and How-To content related to sustainability: tips on the how, where, and why of switching to biological detergents, how to remove stains using natural products, etc...
  • Social media platforms that allow users to load and publish their own content.

  • The same engine used for iSee, but extended to several other products, has been instantiated in a series of installations presented in 2009-2010 at several venues across Europe. Presented for the first time at the Piemonte Share Festival in 2009, Squatting Supermarkets reproduced an interactive supermarket in which the widespread interconnection network defining products’ histories and stories  becomes explicit and accessible. The techniques of traceability and control are transformed into their eco-systemic, narrative and poetic version. Buying a product turns into an immersive experience with regards to its story, and the possibility of writing a part of it, exposing our points of view and our emotions. Products become alive, becoming a space for expression, a network of relations, a domain for possibility and opportunity. The access door to these domains are the logos recognized by the iSee engine, that through Squatting Supermarkets are instantiated in an immersive experience, where the hidden stories of producers and consumers come in contact to foster a new kind of ethical consumerism.

    Fig. 2 Squatting Supermarkets presented at the ToShare Festival in Turin (Italy), 2009.[/caption]

    Technological architecture

    iSee's technological architecture is composed by several horizontal (technologies, sub-systems, protocols, algorithms) and vertical (web and mobile systems; computer vision, social networking, research and interaction algorithms). The following diagram provides a global overview of the architecture:

    Fig. 3 iSee Technological Architecture.

    It is possible to analyze the architecture starting from the top of the diagram. The first module represents the logical components that allow for the creation of access points to iSee's system/ecosystem. As shown in the diagram, the CLIENT system is designed to provide its users and implementers with an abstraction layer through which the functionalities are made accessible on multiple channels: web, mobile and RPC (Remote Procedure Call).

    The client layers invoke functionalities on the rest of the architecture:

  • LOGO services
  • SOCIAL interactive layer
  • I/O services
  • LOGO services include all the algorithmic and system components used to handle the logo acquisition and recognition processes. A processing kernel and a management layer make up this component. The processing kernel combines several strategies to operate on bidimensional images, implementing a generalized system that can also be used to recognize faces and other bidimensional patterns.

    The processing and acquisition process is described by the following diagram:

    Fig. 3 iSee processing and acquisition process.

    A series of images with different framings of the objects are uploaded into the system. Images are processed and brought to comparable levels of contrast, luminosity, white and colour balance, noise levels. Multiple processes are then applied:

    - Speeded Up Robust Feature detection (SURF) [6]
    - Haar classifier feature detection
    - Flood-fill-based and texture-based blob detection

    Images are, thus, transformed into value matrices describing the images' features, their interrelation with the Haar models already present in the system (the logos already recognized by the application) and the Blobs distributions. Matrices are compared, and also scaled and rotated, to add tolerance for different framings of the logos.

    Possible outcomes of this process are:
    - general error, due to insufficient image quality
    - no coupling
    - ambiguous coupling
    - unambiguous coupling with x% tolerance

    In the last two cases the logo is said to be recognized. The second and third case open up the possibility of either adding an additional logo to the database, or extending the definition of a logo that is already present in the system by manually coupling it to the images. In this last case the information present in the database is updated and:
    - the Haar models are created/updated and weighted according to user choices
    - feature recognition systems are tuned
    - blob distribution schemes are updated

    The SOCIAL interactive layer and the SOCIAL graph allow for the management, visualization and representation of social interactions taking place in iSee. Users can use multiple forms of interaction, each of which is mapped using a Content and Social Graph Management System. Content can be multimedia, including texts, sounds, videos, documents and hyperlinks. Each interaction generates interrelations, shaping the social graph. Interrelations can be explicit (for example, by tagging contents) or implicit (for example, whenever people provide content in geographical locations that are close to each other or on the same products). I/O Systems are general purpose software components that handle the distribution and adaptation of multimedia contents to multiple client devices.

    The functionality layer uses the iSee Databases, containing both the information used to perform the aforementioned processes and the meta-information used to access external sources. iSee uses external data source through aggregation, indexing and querying processes. The following sources of information can be processed:
    - external databases
    - websites
    - web services
    - RSS feeds
    - XML databases and derivates (e.g.: KML geographic data)
    - geographic databases and GIS (shapefiles or WMS)

    The social graph is used to estimate multiple semantic contexts that are used to query all the registered information sources, both periodically and on-demand, to respond to user requests. It is in this way that several systems are currently integrated in the iSee platform, including SourceMap (an open source supply chain created at the Massachusetts Institute of Technology), and EarthWatch by NASA and the United Nations, or the Corporate Watch Database. A Web API is also offered to external developers to be used to query the iSee system from external applications.


    Our goal, with the iSee application and Squatting Supermarkets, has been to create a massively distributed process capable of superimposing an informational (and educational) layer to common shopping practices, transforming them into potential occasions for a new ethical social consumerism. Although the iSee and Squatting Supermarkets' engines are at an early stage of development and more thorough research and testing is needed to assess their actual potential, we aim at freely distributing this technological framework so that researchers, designers and people all around the world can actually instantiate it for their own purposes, thus amplifying the power of socially-driven consumerism dynamics.

    De Luca, P. (2006) Il consumo critico: una ricerca esplorativa sulla dimensione sociale del comportamento del consumatore, Congresso Internazionale “Le tendenze del marketing” , Università Ca’ Foscari, Venezia, 20-21 Gennaio 2006.
    Henry Jenkins, H. (2008), Convergence Culture: Where Old and New Media Collide, NYU Press
    Li, C, Bernoff, J. (2006), Groundswell: Winning in a World Transformed by Social Social Technologies, Harvard Business Press.
    Michetti, N., Cosenza, V. 2009 “The Social Consumer” Digital PR / Hill & Knowlton.
    Pine, J. II; Gilmore, J. H (1999) The Experience Economy, USA, Harvard Business Press
    Purcell, K., Rainie, L., Mitchell, A.; Rosenstiel, T., Olmstead, K.(2010) Understanding the participatory news consumer. Pew Research Center.
    Qualman, E. (2009) Socialnomics, John Wiley & Sons, Inc., Hoboken, NJ
    Thackara, J. (2008) In the Bubble: Designing in a Complex World. Cambridge, Mass: MIT Press
    [1] Sustainable Futures, Havas Media 2009 / GFK Eurisco CSR Monitor 2009
    [2 ]Celia, J. “Sustainability and the Shopper”, 55th  Cannes Lions Advertising Festival, Cannes, France, 17th June 2008
    [4] Mintel GNPD Packaging – in-depth insight into packaging innovation: http://www.gnpd.com/sinatra/gnpd/frontpage/?__cc=1 /
    [5] http://www.gartner.com/it/page.jsp?id=985912 // http://www.gartner.com/it/page.jsp?id=1278413
    [6] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded-up robust features. In the 9th European Conference on Computer Vision, Graz, Austria, 2006.