Hypervideo

Hypervideo, or hyperlinked video, is adisplayed video stream that contains embedded, user clickable anchors, allowing navigation between video and other
hypermedia elements. Hypervideo is thus analogous to hypertext,which allows a reader to click on a word in one document and retrieve informationfrom another document, or from another place in the same document.That is, hypervideo combines video with a non linear information structure,allowing a user to make choices based on the content of the video and theuser's interests.

A crucial difference between hypervideo and hypertext is the element of time.Text is normally static, while a video is necessarily dynamic; the content ofthe video changes with time. Consequently, hypervideo has different technical,
aesthetic, and rhetorical requirements than astatic hypertext page. For example,hypervideo might involve the creation of a link from an object in a video thatis visible for only a certain duration. It is therefore necessary to segment thevideo appropriately and add the metadata required to link from frames - oreven objects - in a video to the pertinent information in other media forms.

History of Hypervideo

Illustrating the natural progression to hypervideo from hypertext,the software "Storyspace", [ [http://www.eastgate.com/storyspace/index.html Storyspace: Storyspace ] ] a hypertext writing environment, employs a spatial metaphorfor displaying links. "Storyspace" utilizes 'writing spaces', generic containers forcontent, which link to other writing spaces. "HyperCafe", HyperCafe: Narrative and Aesthetic Properties of Hypervideo, Nitin Nick Sawhney, David Balcom, Ian Smith, UK Conference on Hypertext] a popular experimental prototype of hypervideo, made use of this tool to create"narrative video spaces". "HyperCafe" was developed as an early model of a hypervideosystem, placing users in a virtual cafe where the user dynamically interacts with thevideo to follow different conversations.

Video to video linking was demonstrated by the Interactive Cinema Group at the
MIT Media Lab. "Elastic Charles"] was a hypermedia journal developedbetween 1988 and 1989, in which "micons" were placed inside a video, indicatinglinks to other content. When implementing the Interactive Kon-Tiki Museum, [ Liestol, Gunner. Aesthetic and Rhetorical Aspects of linking Video in Hypermedia] Listol used micons in order to represent video footnotes.Video footnotes were a deliberate extension of the literary footnote appliedto annotating video, thereby providing continuity between traditional textand early hypervideo.] In 1993, Hirata et al. [ Hirata, K., Hara, Y., Shibata, N., Hirabayashi, F., 1993, Media-based navigation for hypermedia systems, in Hypertext '93 Proceedings. ] considered media based navigation for hypermedia systems,where the same type of media is used as a query as for the media to be retrieved.For example, a part of an image (defined by shape, or color, for example) could link toa related image. In this approach, the content of the video becomes the basisof forming the links to other related content.

"HotVideo" was an implementation of this kind of hypervideo, developed at IBM'sChina Research Laboratory in 1996.Cite web|url=http://domino.watson.ibm.com/comm/wwwr_thinkresearch.nsf/pages/news397.html#four|title=New Initiatives - HotVideo: The Cool Way to Link|accessdate=2008-09-30|publisher=IBM|work=Research News] Navigation to associated resources was accomplished by clicking on a dynamic object in a video.In 1997, a student project at the MIT Media Lab called "Hypersoap" further developed this concept."HyperSoap" was a short soap opera program in which a viewer could click with anenhanced remote control on objects in the video to find information on how they couldbe purchased. The company Watchpoint Media was formed in order to commercialize thetechnology involved, resulting in product called Storyteller, oriented towards
interactive television.

Watchpoint Media was acquired by Goldpocket in 2003, which was in turn acquired by Tandberg Television in late 2005.Fact|date=October 2008

In 1997, the Israeli software firm Ephyx Technologies released a product called"v-active",Cite news|url=http://www.byte.com/art/9707/sec16/art2.htm|title=Internet Innovations from Israel |accessdate=2008-10-01|publisher=BYTE|date=July 1997|author=Tania Hershman] the first commercial object based authoring system for hypervideo. Thistechnology was not a success, however; Ephyx changed its name to Veon in 1999,at which time it shifted focus away from hypervideo to the provision of developmenttools for web and broadband content.Cite news|url=http://findarticles.com/p/articles/mi_m0CGN/is_n3400/ai_20538954|title=Ephyx Changes Name To Veon|accessdate=2008-10-01|publisher=Computergram International|date=1998-04-29]

"VideoClix", a hypervideo authoring tool ableto dynamically track and link objects, was released in 2001 byeline Technologies, founded in 1999 as a provider ofhypervideo solutions. [ [http://www.digitalmediadesigner.com/2001/11_nov/news/videoclix011105.htm eLine Launches VideoClix ] ] With the advantage that itsvideos can play back in popular video players such as QuickTime and
Flash,this product has proven to be a commercial success. In 2006, eline Technologies wasacquired by VideoClix Inc.

Concepts and Technical Challenges

Hypervideo is challenging, compared to hyperlinked text, dueto the unique difficulty video presents in node segmentation;that is, separating a video into algorithmicallyidentifiable, linkable content.

Video, at its most basic, is a time sequence of images, which are in turn simply twodimensional arrays of color information. In order to segment a video into meaningfulpieces (objects in images, or scenes within videos), it is necessary to provide acontext, both in space and time, to extract meaningful elements from this imagesequence. Humans are naturally able to perform this task; however, developing amethod to achieve this automatically (or by algorithm) is a complex problem.

And it is desirable to do this algorithmically. At an NTSC frame rate of 30frames per second, [ [http://people.csail.mit.edu/tbuehler/video/ntsc.html NTSC Basics ] ] even a short video of 30 seconds comprises 900 frames.The identification of distinct video elements would be a tedious task if humanintervention were required for every frame. Clearly, even for moderate amountsof video material, manual segmentation is unrealistic.

From the standpoint of time, the smallest unit of a video is the frame(the finest time granularity). Node segmentation could be performed at the frame level - a straightforward task as a frame is easily identifiable.However, a single frame cannot contain video information, since videos arenecessarily dynamic. Analogously, a single word separated from a textdoes not convey meaning. Thus it is necessary to considerthe scene, which is the next level of temporal organization. A scene can be definedas the minimum sequential set of frames that conveys meaning. This is an importantconcept for hypervideo, as one might wish a hypervideo link to be active throughoutone scene, though not in the next. Scene granularity is therefore natural in thecreation of hypervideo. Consequently, hypervideo requires algorithms capable ofdetecting scene transitions.

Of course, one can imagine coarser levels of temporal organization. Scenes canbe grouped together to form a narrative sequence, which in turn aregrouped to form a video; from the point of view of node segmentation, these conceptsare not as critical. Issues of time in hypervideo were considered extensively in thecreation of the "HyperCafe".

Even if the frame is the smallest time unit, one can still spatially segment a video ata sub-frame level, separating the frame image into its constituent objects;this is necessary when performing node segmentation at the objectlevel. Time introduces complexity in this case also, for even after an object isdifferentiated in one frame, it is usually necessary to follow thesame object through a sequence of frames. This process, known as object tracking,is essential to the creation of links from objects in videos. Spatialsegmentation of object can be achieved, for example, through the use of intensitygradients to detect edges, color histograms to match regions, Smith, Jason and Stotts, David, An Extensible Object Tracking Architecture for Hyperlinking in Real-time and Stored Video Streams, Dept. of Computer Science, Univ. of North Caroline and Chapel Hill] motion detection, [ Khan, Sohaib and Shah, Mubarak, Object Based Segmentation of Video Using Color, Motion and Spatial Information, Computer Vision Laboratory, University of Central Florida] or a combination of these and other methods.

Once the required nodes have been segmented and combined with the associated linkinginformation, this metadata must be incorporated with the original video forplayback. The metadata is placed conceptually in layers, or tracks, on top ofthe video; this layered structure is then presented to the user for viewing andinteraction. Thus the display technology, the hypervideo player, should not beneglected when creating hypervideo content.For example, efficiency can be gained by storing the geometry of areas associatedwith tracked objects only in certain keyframes, and allowing the player to interpolatebetween these keyframes, as developed for "HotVideo" by IBM. [ U.S. Patent 6912726] Furthermore, the creators of "VideoClix" emphasize the fact that its contentplays back on standard players, such as Quicktime and Flash.When one considers that the Flash player alone is installed on over 98% of internetenabled desktops in mature markets, [ [http://www.adobe.com/products/player_census/flashplayer/ Adobe - Flash Player Statistics ] ] this a perhaps a reason for the success of this product in the current arena.

Hypervideo authoring tools

The process of creating hypervideo content is known as authoring.Many early attempts at creating widely distributed authoring tools were not successful,for a variety or reasons. However, this field is currently enjoying a resurgence ofinterest, perhaps due to the greater availability of broadband internet.

"VideoClix", described on its website as the premier and onlycommercially availabletechnology for creating clickable videos. It is prominent in therapidly growing domain of internet video. Tandberg Television, specializing ininteractive television solutions, has a hypervideo system called "AdPoint"for video-on-demand. They also sell "Storyteller", a product derived from theMIT project "Hypersoap".

"Asterpix" offers a hypervideo service and browser. Users can convert internet videos from popular sites such as YouTube, Dailymotion, Myspace etc into hypervideo on the fly and viewers may interact and navigate the hypervideo without the need for any special software. In addition, the Asterpix robot "asterbot" automatically converts internet videos from popular sites into interactive video by tagging the most relevant objects in each video.

"Klickable" provides a simple web based tool to annotate videos. Klickable technology allows content publishers to create hotspots within the video, add a photo and link to wherever they choose. Klickable videos, create a more engaged user, a comprehensive publishing experience and targeted contextual advertising.

"Adivi" (Add Digital Information to VIdeo) is a project of the
Darmstadt University of Technology, Germany. They are studying the potentialof hypervideo to support collaborative documentation. Siemens,an engineering firm, will use this technology for enhanced on-line training manuals.

Adobe Flash, a popular multimedia authoring program widely used to create animatedweb content, can also be used to create hypervideo content. As Flash was not designedas a hypervideo authoring tool, creating such content can be difficult using Flashalone. Such added functionality has been provided through outside software in thepast - for example, "MoVideo" and "Digital Lava".However, these products are no longer sold.

Riva Producer is a software that is especially designed to reduce production costs of non-linear video navigation. Therefore it is suitable to replace industrial documentation by utility films.

In the past, there have been a number of attempts to market hypervideo authoringsoftware that is no longer available."MediaLoom", [ [http://www.mindspring.com/~jntolva/medialoom/index.html hypertext :: renaissance is now Ascent Stage ] ] a product based on a Masters of Science project at the
Georgia Institute of Technology, was an early hypervideo authoring tool.It used the "Storyspace" hypertext authoring environment to generate scriptfiles for the hypervideo engine of the "HyperCafe". This product reachedprototype stage, but was not commercially successful.Ephyx Technologies created "v-active", the first authoring software usingdynamically tracked objects in video. The company moved away from hypervideo,however, when it became Veon in 1999.

Hypervideo can also be created using services provided by firms with proprietarymethods, such as those provided by Vimation. However, this company does notlicence its authoring software.

The rise of hypervideo

As the first steps in hypervideo were taken in the late 1980s, it would appearthat hypervideo is taking unexpectedly long to realize its potential.Many interesting experiments ("HyperCafe", "HyperSoap") have not been extensivelyfollowed up on, and authoring tools are at the moment available from onlya small number of providers.

However, perhaps with the wider availability of broadband internet,this situation is rapidly changing. Interest in hypervideo is increasing,as reflected in popular blogs on the subject, [ [http://www.solitude.dk/ Andreas Haugstrup Pedersen | solitude.dk ] ] [ [http://www.viewmagazine.tv/Video%20hyperlinking.html Hyper text, now video hyperlinking ] ] as well as the extraordinary rise of the internet phenomenon YouTube.Furthermore, by 2010, some estimates have internet downloads claimingover one third of the market for on-demand video. [ The Economist, Feb. 8 2007, What's on next ]

As the amount of video content increases and becomes available on the internet,the possibilities for linking video increase even faster. Digital libraries areconstantly growing, of which video is an important part. News outlets haveamassed vast video archives, which could be useful in education and historicalresearch. Direct searching of pictures or videos,a much harder task then indexing and searching text,could be greatly facilited by hypervideo methods.

Perhaps the most significant consequence of hypervideo will result from
commercial advertising. Devising a business model tomonetize video hasproven notoriously difficult. The application of traditional advertisingmethods - for example introducing ads into video - is likely to be rejectedby the online community, while revenue from selling advertising on videosharing sites has so far not been promising. [ The Economist, Aug 31st 2006, The trouble with YouTube.] Hypervideo offers an alternate way to monetize video, allowing for the possibility ofcreating video clips where objects link to advertising or e-commerce sites, or providemore information about particular products. This new model of advertising is lessintrusive, only displaying advertising information when the user makes the choice byclicking on an object in a video. And since it is the user who has requested theproduct information, this type of advertising is better targeted and likely to be moreeffective.

Notes

Further reading

* [http://www.economist.com/displaystory.cfm?story_id=E1_SJDQTGG From hypertext to hypervideo]
* [http://www.csdl.tamu.edu/~l0f0954/academic/cpsc610/p-1.htm A Picture of Hypervideo Today]
* [http://www.patentstorm.us/patents/6462754.html U.S. Patent 6462754]
* [http://rockfish.cs.unc.edu/pubs/TR02-017.pdf Technical Report TR02-017 UNC Chapel Hill]

External links

* [http://www.media.mit.edu/hypersoap/ Hypersoap page at MIT]
* [http://www.videoclix.com VideoClix]
* [http://www.eastgate.com/storyspace/index.html Storyspace]
* [http://www.asterpix.com Asterpix]