The Potentials of Google Vision API-based Networks to Study Natively Digital Images

Main Article Content

Janna Joceli Omena
Pilipets Elena
Beatrice Gobbo
Chao Jason


In this article, we present the potentials of Google Vision API-based networks for studying online images, covering three important modalities as part of a critical visual methodology: the content of the image itself, its specific ‘audiencing’ through web references (or image metadata), and the sites of image circulation. First, we conceptually and technically define different networks built upon computer vision features: image-label, image-web entities, and image-domain. Second, we present a research protocol diagram that illustrates how to build networks of images and respective descriptions or sites of circulation. Third, we discuss the potentialities of computer vision networks as a research device, stressing their data-relational (trans)formations and interpretative specifics. Three different case studies will be introduced as examples. In conclusion, we argue that such a visual methodology requires critical technical practices accounting for the multiple layers of technical mediation involved.

Article Details

How to Cite
Omena, J. J., Elena , P., Gobbo, B., & Jason , C. (2021). The Potentials of Google Vision API-based Networks to Study Natively Digital Images. Diseña, (19), Article.1.
Original articles
Author Biographies

Janna Joceli Omena, Center for Advanced Internet Studies (CAIS)

Master in Contemporary Culture and New Technologies, Universidade NOVA de Lisboa. Research fellow at the Center for Advanced Internet Studies (CAIS). She is a member of iNOVA Media Lab and the Public Data Lab. Her research focuses on digital methods, digital network studies, and technicity-of-the-mediums in support of social and medium research. She is the editor of Métodos Digitais: Teoria-Prática-Crítica (ICNOVA, 2019) and the coordinator of the SMART Data Sprint. Some of her latest publications are: ‘Digital Methods for Hashtag Engagement Research’ (with E.T. Rabello and A.G. Mintz; Social Media + Society, Vol. 6, Issue 3) and ‘Call into the Platform!’ (with A. Granado; Icono14, Vol. 18, Issue 1).

Pilipets Elena , Universität Klagenfurt, Department of Media and Communications

Doctor in Media Studies, Universität Klagenfurt. Postdoc researcher at the Depart­ment of Media and Communications, Universität Klagenfurt, and SMART (Social Media Research Techniques) researcher with iNOVA Media Lab, Universidade NOVA de Lisboa. Her teaching and research interests are related to media cultural studies, internet research, and digital methods. She is the current holder of the 2020 Carinthian Award for Young Social Sciences and Humanities Scholars. Her recent publications include ‘Nipples, Memes, and Algorithmic Failure: NSFW Critique of Tumblr Censorship’ (with S. Paasonen; New Media & Society, 1st Publ., 2020) and ‘Digitale Medien und Methoden. Über den methodischen Umgang mit visuellen Pla­ttforminhalten und Internet-Memes’ (Open-Media- Studies-Blog der Zeitschrift für Medienwissenschaft).

Beatrice Gobbo, Politecnico di Milano, DensityDesign Lab

Master in Communication Design, Politecnico di Milano. Ph.D. student in Design at Politecnico di Milano. She is a member of the Density­Design Lab, a research group focused on information visualization and information design. Her current research is focused on the role of communication design and information visualization in the field of explainable artificial intelligence. Some of her recent publications include ‘Research Protocol Diagrams as Didactic Tools to Act Critically in Dataset Design Pro­cesses’ (with M. Mauri, M. A. Briones, and G. Colom­bo; in INTED2020 Proceedings) and ‘Explaining AI through Critical Reflection Artifacts. On the Role of Communication Design within XAI’ (in T. Reis, M.X. Bornschlegl, M. Angelini, and M.L. Hemmje, Eds.; Advanced Visual Interfaces. Supporting Artificial Inte­lligence and Big Data Applications; Springer, 2020).

Chao Jason , Universität Siegen, Collaborative Research Centre "Media of Cooperation"

Master in Big Data and Digital Futu­res, University of Warwick. Master in Human Rights Law, University of London. He is a researcher and PhD candidate at the Universität Siegen. His current research is focused on the development of digital methods tools for the study of sensor media. He has backgrounds in software development and human rights advocacy. Some research software recently developed by him include: AppTraffic, to study the network traffic of mobile applications; and Offline Image Query and Extraction Tool (with J. J. Omena).


Azar, M., Cox, G., & Impett, L. (2021). Introduction: Ways of Machine Seeing. AI & Society.

Barreto, M. L., Barral-Netto, M., Stabeli, R., Almeida-Filho, N., Vasconcelos, P. F. C., Teixeira, M., Buss, P., & Gadelha, P. E. (2016). Zika Virus and Microcephaly in Brazil: A Scientific Agenda. The Lancet, 387(10022), 919–921.

Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An Open Source Software for Exploring and Manipulating Networks. Proceedings of the International AAAI Conference on Web and Social Media, 3(1), 361–362.

Berners-Lee, T. (1995). Hypertext and Our Collective Destiny [Video]. 1995 Vannevar Bush Symposium - Videos and Links - Doug Engelbart Institute. Retrieved September 20, 2020, from

Chao, J. (2021). Memespector GUI: Graphical User Interface Client for Computer Vision APIs [Computer Software] (Version 0.1) [C#] (Original work published 2021).

Colombo, G. (2018). The Design of Composite Images. Displaying Digital Visual Content for Social Research [Doctoral Dissertation, Politecnico di Milano].

d’Andréa, C., & Mintz, A. (2019). Studying the Live Cross-Platform Circulation of Images with Computer Vision API: An Experiment Based on a Sports Media Event. International Journal of Communication, 13, 1825–1845.

Garimella, K., & Eckles, D. (2020). Images and Misinformation in Political Groups: Evidence from WhatsApp in India. Harvard Kennedy School Misinformation Review, 1(5).

Geboers, M. A., & Van De Wiele, C. T. (2020). Machine Vision and Social Media Images: Why Hashtags Matter. Social Media + Society, 6(2), 2056305120928485.

Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2020). Datasheets for Datasets. ArXiv, 1803.09010 [cs].

Gerlitz, C., & Rieder, B. (2018). Tweets Are Not Created Equal: Investigating Twitter’s Client Ecosystem. International Journal of Communication, 12.

Gibbs, M., Meese, J., Arnold, M., Nansen, B., & Carter, M. (2015). #Funeral and Instagram: Death, Social Media, and Platform Vernacular. Information, Communication & Society, 18(3), 255–268.

Google Cloud. (2017, April 13). Pricing | Google Cloud Vision API Documentation. Wayback Machine.

Google User Content. (2020). General Guidelines.

Hochman, N. (2014). The Social Media Image. Big Data & Society, 1(2), 2053951714546645.

Hoelzl, I., & Marie, R. (2015). Softimage: Towards a New Theory of the Digital Image. Intellect.

Jacomy, M. (2013). Table 2 Net [Computer Software].

Jacomy, M. (2019). The Web as Layers [Video]. HYPHE - Session 4.

Jacomy, M. (2021). Situating Visual Network Analysis [Doctoral Dissertation, Aalborg University].

Jacomy, M., Venturini, T., Heymann, S., & Bastian, M. (2014). ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi Software. PLOS ONE, 9(6), e98679.

Leetaru, K. (2019, June 1). Using Google Vision AI’s Reverse Image Search To Richly Catalog Television News. Forbes.

MacKenzie, A., & Munster, A. (2019). Platform Seeing: Image Ensembles and their Invisualities. Theory, Culture & Society, 36(5), 3–22.

Manovich, L. (2020). Cultural Analytics. MIT Press.

Mauri, M., Briones, M. A., Gobbo, B., & Colombo, G. (2020). Research Protocol Diagrams as Didactic Tools to Act Critically in Dataset Design Processes. INTED2020 Proceedings, 9034–9043.

Mintz, A., Silva, T., Gobbo, B., Pilipets, E., Azhar, H., Takamitsu, H., Omena, J. J., & Oliveira, T. (2019). Interrogating Vision APIs. #SMARTDataSprint.

Niederer, S., & Colombo, G. (2019). Visual Methodologies for Networked Images: Designing Visualizations for Collaborative Research, Cross-platform Analysis, and Public Participation. Diseña, (14), 40–67.

Omena, J. J. (in press). Digital Methods and Technicity-of-the-Mediums. From Regimes of Functioning to Digital Research [Doctoral Dissertation]. Universidade NOVA de Lisboa.

Omena, J. J., & Amaral, I. (2019). Sistema de leitura de redes digitais multiplataforma. In J. J. Omena (Ed.), Métodos Digitais: Teoria‐prática‐crítica (pp. 121–140). ICNOVA.

Omena, J. J., & Granado, A. (2020). Call into the Platform! Revista ICONO14 Revista Científica de Comunicación y Tecnologías emergentes, 18(1), 89–122.

Omena, J. J., Rabello, E. T., & Mintz, A. G. (2020). Digital Methods for Hashtag Engagement Research. Social Media + Society, 6(3), 2056305120940697.

Paglen, T. (2014, March 13). Seeing Machines. Is Photography Over? Fotomuseum Winterthur Series.

Parikka, J., & Dvořák, T. (2021). Introduction: On the Scale, Quantity and Measure of Images. In T. Dvořák & J. Parikka (Eds.), Photography Off the Scale: Technologies and Theories of the Mass Image (pp. 1–24). Edinburgh University Press.

Pearce, W., Özkula, S. M., Greene, A. K., Teeling, L., Bansard, J. S., Omena, J. J., & Rabello, E. T. (2020). Visual Cross-platform Analysis: Digital Methods to Research Social Media Images. Information, Communication & Society, 23(2), 161–180.

Pilipets, E., Flores, A. M. M., Flaim, G., Skazedonig, M., Sepúlveda, R., & Del Nero, S. (2020). From “Tumblr Purge” to “Female Nipples”: Telling a Story of Platform Censorship Critique through Memes and Digital Methods. #SMARTDataSprint.

Pilipets, E., & Paasonen, S. (2020). Nipples, Memes, and Algorithmic Failure: NSFW Critique of Tumblr Censorship. New Media & Society, 1461444820979280.

Rettberg, J. W. (2020). Situated Data Analysis: A New Method for Analysing Encoded Power Relationships in Social Media Platforms and Apps. Humanities and Social Sciences Communications, 7, Art. 5.

Ricci, D., Colombo, G., Meunier, A., & Brilli, A. (2017). Designing Digital Methods to Monitor and Inform Urban Policy: The Case of Paris and its Urban Nature Initiative. International Conference on Public Policy (ICPP3).

Rieder, B. (2015). YouTube Data Tools [Computer Software] (Version 1.11).

Rieder, B. (2020). Engines of Order: A Mechanology of Algorithmic Techniques. Amsterdam University Press.

Rieder, B., & Röhle, T. (2018). Digital Methods: From Challenges to Bildung. In M. T. Schäfer & K. van Es (Eds.), The Datafied Society: Studying Culture through Data (pp. 109–124). Amsterdam University Press.

Robinson, S. (2017, June 1). Exploring the Cloud Vision API. Medium.

Rogers, R. (2013). Digital Methods. MIT Press.

Rogers, R. (2019). Doing Digital Methods. SAGE.

Rose, G. (2016). Visual Methodologies (4th ed.). Open University.

Rubinstein, D., & Sluis, K. (2013). Concerning the Undecidability of the Digital Image. Photographies, 6(1), 151–158.

Schwemmer, C., Knight, C., Bello-Pardo, E. D., Oklobdzija, S., Schoonvelde, M., & Lockhart, J. W. (2020). Diagnosing Gender Bias in Image Recognition Systems. Socius, 6, 2378023120967171.

Silva, T., Barciela, P., & Meirelles, P. (2018). Mapeando Imagens de Desinformação e Fake News Político-Eleitorais com Inteligência Artificial. 3o CONEC: Congresso Nacional de Estudos Comunicacionais Da PUC Minas Poços de Caldas - Convergência e Monitoramento, 413–427.

Silva, T., Mintz, A., Omena, J. J., Gobbo, B., Oliveira, T., Takamitsu, H. T., Pilipets, E., & Azhar, H. (2020). APIs de Visão Computacional: Investigando mediações algorítmicas a partir de estudo de bancos de imagens. Logos, 27(1), 25–54.

Steyerl, H. (2009). In Defense of the Poor Image. E-Flux Journal, 10.

Sullivan, D. (2020, May 20). A Reintroduction to our Knowledge Graph and Knowledge Panels. Blog.Google.

Szeliski, R. (2021). Computer Vision: Algorithms and Applications (2nd ed.). Springer.

Taibi, D., Rogers, R., Marenzi, I., Nejdl, W., Ahmad, Q. A. I., & Fulantelli, G. (2016). Search as Research Practices on the Web: The Sar-web Platform for Cross-language Engine Results Analysis. Proceedings of the 8th ACM Conference on Web Science, 367–369.

Tumblr. (2018, December). Updates to Tumblr’s Community Guidelines [Tumblr]. Tumblr Support.

Venturini, T., Jacomy, M., & Jensen, P. (2019). What Do We See When We Look at Networks. An Introduction to Visual Network Analysis and Force-Directed Layouts (SSRN Scholarly Paper ID 3378438). Social Science Research Network.

Venturini, T., Jacomy, M., & Pereira, D. (2015). Visual Network Analysis: The Example of the Rio+20 Online Debate.

Xi, N., Ma, D., Liou, M., Steinert-Threlkeld, Z. C., Anastasopoulos, J., & Joo, J. (2020). Understanding the Political Ideology of Legislators from Social Media Images. Proceedings of the 14th International AAAI Conference on Web and Social Media, ICWSM 2020, 726–737.