Abstract
This paper is the textual counterpart of the presentation made at FCUL, in 2003, in the seminars organized by the “Encyclopaedia and Hypertext Project”. The paper, and presentation, travels through the issues of information spaces, basic mechanisms for information structuring, meaning creation, interpretation and open issues.
Keywords: Information, Segmentation, Indexing, Interpretation and Meaning
1 - Introduction
Current computing systems and environments are organized according to a conceptual framework developed around a number of high level concepts and relations. This framework provides the foundations for the intellectual and cognitive nature of current and future computing systems [Levy 90].
Information processing and computing capacity operate together with the cognitive features of human beings to produce “augmented systems” that allow us to create, manipulate and share knowledge. The notion of augmentation has been presented back in the 60’s by Doug Engelbart [Engelbart 88]. Together with the notion of “structural coupling” presented in [Winograd 86] and [ Maturana 87 ], which defines the actual conditions of augmentation, new ways of creating, thinking and acting are enabled.
The emergence of the notion of hypertext and hypermedia shed light on the relevance of the information structure as a fundamental mechanism for the management of the information semantics. Hypertext and hypermedia systems create this opportunity through the manipulation and management of the information structure [Delany 91].
As with other media and information environments, the reconfiguration and combination of information elements is the fundamental process for the creation of meaning.
One can state that the progressive fragmentation of a “coherent” body of information tends to de-localize the semantics of the information from the parts to the structure up to the point where we admit that the only relevant source of meaning is the information structure – the relations between information atoms – rather then the actual content of the atoms themselves. As examples one can compare a large poem with its isolated words, the Pyramids and its stone blocks, chemical molecules and ADN.
2 - Segmentation and Indexing
In the creation and manipulation of hypermedia information spaces, some recurring topics and themes can be identified: the capture and discovery of the information “structure”, together with the construction-deconstruction-reconstruction cycles; the identification, separation and management of the meta-information, or metadata, information that describes the information; and the discovery and identification of (possibly hierarchically organized) units of meaning.
The creative opportunities and the richness of the information spaces are enabled by a generic cycle of segmentation, indexing, combination and interaction.
Segmentation is based on the identification or extraction of features of the particular media that lead to the determination of atomic elements. The atomicity of information elements is clearly dependent of the context of use and combination.
Indexing corresponds to the tagging of the information elements based on any classification criteria.
Original “hypertext” systems were based on a default and basic segmentation – the textual words or other units able to be transformed in “anchors”. Indexing has been progressively introduced through the development of the data models supporting the enhancement of the information structure with additional attributes.
Computationally, the above processes materialize in specific mechanisms and techniques. Segmentation is performed through the processing or parsing of the digitally encoded information in a compressed or uncompressed state. The parsing techniques search for significant features encoded in the information based on physical characteristics identified in the source information. Sample features are “silence”, “a given speaker”, “a musical sequence” (in audio information), a “scene change”, a “camera zoom or pane”, a “face” (in video information), and the techniques used develop a range of pattern matching mechanisms, such as colour variation analysis, movement vector analysis, etc.. [Hampapur 94].
The initial segmentation approaches usually address syntactical notions (phrases, scenes, spots) rather that events that have a strong semantic charge , such as a “battle”, a “sad scene” or “joke”. These high level semantic features require some mapping from elementary physical characteristics to a footprint of features that are present in a particular semantic event.
Indexing requires the super-imposition of one (or several) “schema” that defines the classification of elements This classification, for description and search, follows the vocabulary defined in the schema and its particular relations (hierarchical relations for example) [Chai 03].
The technology that has become the widespread solution for information indexing is based on the markup languages (e.g. SGML). Currently XML - eXtensible Markup Language and derived languages (MathML, CML, ...) support the enhancement of raw information with a flexible mechanism to express – and share - structure.
Markup languages in general, and XML-based descriptions in particular, define entities, associated attributes and reciprocal relations (actually a lexicon and a syntax). Other advantages of this approach are interoperability, integration with DBMS’s, separation between data and presentation, etc.
3 - Repurposing, Redesign, Rethorics and Interpretation
Once segmentation and indexing occurs, up to some level of detail and based on a given set of features and classification options, combination and interaction processes are enabled. In early hypertext systems, this combination happens whenever “links” are created. In current
rich media environments , the principles are similar: the combination processes are carried out with the purpose of telling stories through the establishment of relations between the elements, that regulate and lead to the production of meaning. Examples of related activities are staging, montage, sound editing, …[ Davis 94], [Sparacino 00].
Once media elements are defined, indexed and combined and available for interaction, new rethorical opportunities, for both authors and readers are available [Liestøl 94]. This flexibility enhances the richness of our information spaces where the semiotics and interpretation theories gain an added relevance and aplicability [Eco 90] [Eco 92].
4 - Other Windows
The creation and perception of meaning in digital information spaces cannot be separated from the social and physical experience. This reciprocity is analogous and related with the association between language and action strongly evident in the speech act theory
[Searle69], actually applied in concrete computing systems supporting cooperative activities [Auramaki 88],
The physical configuration of traditional computing systems has shown its limitations and induced the design and exploitation of new ways of interaction, not imposing such large cognitive constraints. The “tablet PC” that is being experimented by more and more users, and other “disappearing computer” initiatives are a clear sign of this evolution [Norman 99].
Fig1 – News ways of computing
The so-called “Information Appliances” [Bergman 00], foundation elements of mobile, ubiquitous or ambient computing, generate opportunities for new structures of information, interaction and interpretation (see fig. 1 above) [Streitz 03].
5 - Undiscovered Meanings
The manipulation of information structures as a source for meaning also applies in domains that are now being “mapped onto” the digital world. This “digitization” of otherwise closed systems creates opportunities for analysis of syntactic an semantic relations and for speculation on the fundamental constituents of the meaning.
Fig 2 – Mapping of Aminoacids onto musical notes
An inspiring and creative example is the world of “bio music” experiments. Based on the digital (discrete) description of the molecular constituents of Genes or Proteins (ACTG in the first case, and a whole alphabet of aminoacids in the second case), one can map biomolecules onto musical notes, and therefore produce “musical plays” out of biological structures (http://www.whozoo.org/mac/Music/samples.htm) – see fig.2 above [Hayashi84], [Munakata 95], [Munakata 97].
If we can, or want to, attribute some meaning to the music thus generated, be it just “aesthetical” meaning or anything more, then, we have to conclude that it emerged from the structure of biological molecules. What this conclusion means, it remains to be seen.
References
| [Auramaki88] | Esa Auramäki, Erkki Lehtinen, Kalle Lyytinen, A speech-act-based office modeling approach, ACM Transactions on Information Systems (TOIS), Volume 6 Issue 2, April 1988 |
| [Baecker87] | R.M. Baecker and W.A. Buxton, Readings in Human-Computer interaction, Morgan Kaufmann, 1987 |
| [Bergman 00] | Eric Bergman (ed), Information Appliances and Beyond, Morgan Kaufman, 2000 |
| [Chai 03] | Wei Chai, Barry Vercoe, Music and digital libraries: from users to algorithms: Structural analysis of musical signals for indexing and thumbnailing, Proceedings of the third ACM/IEEE-CS joint conference on Digital libraries, May 2003 |
| [Davis 94] | Marc Davis, Media streams: representing video for retrieval and repurposing, Proceedings of the second ACM international conference on Multimedia, October 1994 |
| [Delany 91] | P. Delany and G.P. Landow, Hypermedia and Literary Studies, MIT Press, 1991 |
| [Eco 90] | Umberto Eco, I Limiti dell’Interpretazione, G.E.Fabbri, Bompiani, Italy, 1990 |
| [Eco 92] | Umberto Eco, Interpretation and Overinterpretation, Cambridge University Press, 1992. |
| [Engelbart 88] | D. Engelbart and W. English. A Research Center for Augmenting Human Intellect. In Irene Greif, editor, Computer Supported Collaborative Work : A Book of Readings, 81-103, Morgan Kaufman Publishers, 1988 |
| [Hampapur 94] | A. Hampapur, T. Weymouth, R. Jain, Digital video segmentation, Proceedings of the second ACM international conference on Multimedia, October 1994 |
| [Hayashi84] | K. Hayashi and N. Munakata, Basically Musical. Nature, 310, 96 (1984) |
| [Munakata 95] | N. Munakata and K. Hayashi, Gene Music; Tonal assignments of Bases and Amino Acids. pp. 72-83. In Visualizing Biological Information (Ed. C. Pickover), World Scientific, Singapore (1995). (ISBN 9810214278) |
| [Munakata 97] | N. Munakata, Musical Representation of Gene Sequences. pp. 73-82. In Arts Medicine (Eds. R. R. Pratt and Y. Tokuda), MMB Music, Saint Louis (1997). (ISBN 0-918812-92-5). See also Gene Sequence Analysis with Auditory Display. http://www.whozoo.org/mac/Music/Primer/Primer_index.htm |
| [Levy 90] | P. Levy, Les Technologies de l'Intelligence, Éditions La Découverte, 1990 |
| [Liestøl 94] | Gunnar Liestøl, Aesthetic and rhetorical aspects of linking video in hypermedia Proceedings of the 1994 ACM European conference on Hypermedia technology, September 1994 |
| [Maturana 87] | H.Maturana and F. Varela, The Tree of Knowledge, Shambhala Press,1987 |
| [Norman 99] | Donald A. Norman, The Invisible Computer, MIT Press, 1999 |
| [Ricoeur 76] | Paul Ricoeur, Interpretation Theory: discourse and the surplus of meaning, Texas Christian University Press, 1976 |
| [Searle 69] | John Searle, Speech Acts, Cambridge University Press, 1969 |
| [Sparacino 00] | Sparacino F; Davenport G; Pentland A, Media in performance: Interactive spaces for dance, theater, circus, and museum exhibits , IBM Systems Journal, v. 39 no. 3-4, 2000, p. 479-510. |
| [Streitz 03] | Norbert Streitz et all, Ambient Displays and Mobile Devices for the Creation of Social Architectural Spaces: Supporting informal communication and social awareness in organizations. In: K. O’Hara, M. Perry, E. Churchill, D. Russell (Ed.): Public and Situated Displays: Social and Interactional Aspects of Shared Display Technologies, Kluwer Publishers, 2003. pp. 387-409. |
| [Winograd 86] | T.Winograd and F.Flores, Understanding Computers and Cognition, Addison Wesley, 1986 |