|
Digital Camera Patent Abstract
A digital camera has output ports that are connectable by cables
to an HDTV television. The camera generates a slide show viewable
on the television screen. The slide show involves a sequence of
images stored on the camera and audio stored on the camera. The
slide show is supplied to the television in the form of an HDTV
resolution video stream and an accompanying audio stream. A user
selects one of a plurality of scenarios for the slide show. The
particular scenario determines how identified images will be presented.
Using camera buttons, the user can stop and start the slide show
and can move a pointer on the television screen. A standard EVJ
file that defines the slide show in accordance with a scenario is
generated by the camera. The file can be played on any rendering
device that supports the EVJ functionality so as to recreate the
slide show.
Digital Camera Patent Claims
1. A method comprising: receiving onto a digital camera a selection
of one of a plurality of slide show scenarios, wherein the digital
camera stores a plurality digital still images and also stores audio
information; and outputting the selected slide show scenario from
the digital camera as a video stream and an accompanying audio stream,
the video stream including a sequence of the digital still images,
the audio stream including the audio information, wherein the selected
slide show scenario determines how each of the digital still images
in the video stream is panned, zoomed and tilted.
2. The method of claim 1, wherein the digital camera stores a plurality
of sequence sets, wherein each sequence set determines how an associated
plurality of digital still images will be panned, zoomed and tilted,
and wherein each of the plurality of slide show scenarios includes
a different sequence of a different subset of the sequence sets.
3. The method of claim 1, wherein the video stream is a stream
of high definition television (HDTV) video that is communicated
from the digital camera to a television and is displayed as a slide
show on the screen of the television.
4. The method of claim 1, wherein the digital camera stores digital
still images in addition to the plurality of digital still images,
the method further comprising: receiving onto the digital camera
an indication of which of the digital still images stored on the
digital camera will be output from the digital camera in the video
stream.
5. The method of claim 1, further comprising: displaying on a screen
of the digital camera a plurality of digital image file indicators,
a plurality of audio file indicators, and a plurality of scenario
indicators; and receiving from a user of the digital camera a selection
of at least one of the displayed digital image file indicators,
a selection of at least one of the audio file indicators, and a
selection of one scenario indicator, wherein digital images associated
with the selected digital image file indicators and audio information
associated with the selected audio file indicators are output from
the digital camera in accordance with a scenario identified by the
scenario indicator.
6. The method of claim 3, wherein the digital camera includes a
button, the method further comprising: detecting that the button
has been pressed a first time and in response thereto stopping the
sequence of the digital still images in the video stream such that
the slide show displayed on the screen of the television halts;
and detecting that the button has been pressed a second time and
in response thereto starting the sequence of the digital still images
in the video stream such that the slide show displayed on the screen
of the television resumes.
7. The method of claim 3, further comprising: receiving pointer
manipulation input onto the digital camera from a user when the
slide show is being displayed on the screen of the television; and
in response to receiving said pointer manipulation input embedding
pointer information in the video stream such that a pointer is displayed
on the screen of the television, the pointer being movable by the
user.
8. The method of claim 1, further comprising: the digital camera
detecting a location of a face in one of the digital still images;
and the digital camera using the location to control how the digital
still image is zoomed, or panned, or tilted.
9. The method of claim 3, further comprising: receiving input from
a user into the digital camera and in response thereto stopping
the sequence of the digital still images in the video stream such
that the slide show displayed on the screen of the television halts
and such that one of the digital images appears on the screen of
the television; and receiving input from a user to customize a way
that the digital image will appear in the sequence of digital still
images when the digital camera outputs the plurality of digital
still images in the selected slide show scenario from the digital
camera a subsequent time; and outputting the plurality of digital
still images in selected slide show scenario the subsequent time.
10. The method of claim 9, wherein the user customizes the way
that the digital image is zoomed by changing a zoom point.
11. The method of claim 1, further comprising: the digital camera
detecting a frequency component in the audio information; and the
digital camera using the frequency component to control when successive
ones of the digital still images appear in the video stream.
12. The method of claim 1, further comprising: receiving second
audio information into the digital camera via a microphone of the
digital camera; and incorporating the second audio information into
the audio stream.
13. The method of claim 1, wherein the digital camera includes
a wireless transceiver, the method further comprising: receiving
digital still image information onto the digital camera from a wireless
device, wherein the digital still image information is transmitted
from a wireless transceiver of the wireless device and is received
by the wireless transceiver of the digital camera; and incorporating
the digital still image information into the video stream as one
of the digital still images.
14. The method of claim 1, wherein the digital camera includes
a zoom engine, the zoom engine being used to capture a digital still
image, and wherein the zoom engine is also used to zoom digital
still images to create the video stream that is output from the
digital camera.
15. The method of claim 1, further comprising: displaying indications
of the plurality of slide show scenarios on a display of the digital
camera, wherein the selection is received onto the digital camera
in response to a user of the digital camera pressing a button to
select one of the displayed indications.
16. The method of claim 1, wherein some of the digital still images
in the video stream are not panned, are not zoomed and are not tilted.
17. The method of claim 1, wherein the digital camera generates
the video stream in real time from the plurality of digital still
images stored on the digital camera and wherein the digital camera
generates the audio stream in real time from the audio information
stored on the digital camera.
18. The method of claim 1, further comprising: receiving onto a
digital camera a selection of one of a plurality of video clips;
and incorporating the selected video clip into the video stream,
the selected slide show scenario determining how the video clip
is incorporated into the video stream.
19. The method of claim 1, further comprising: downloading the
audio information from a computer into the digital camera.
20. An integrated circuit comprising: a data port coupled to a
storage device, the storage device storing a plurality of digital
still images; a video output port; a digital imaging pipeline; a
zoom engine; and a processor that operates in response to a selected
slide show scenario and that controls the zoom engine such that
selected ones of the digital still images are read from the storage
device into the integrated circuit, the selected digital still images
being output from the video output port as a video stream, wherein
a first of the digital still images is operated on by the zoom engine
such that the video stream includes a sequence of successively more
and more zoomed versions of the first digital still image, wherein
a second of the digital still images is operated on by the zoom
engine such that the video stream includes a sequence of successively
more and more panned versions of the second digital still image,
and wherein a third of the digital still images is operated on by
the zoom engine such that the video stream includes a sequence of
successively more and more tilted versions of the third digital
still image.
21. The integrated circuit of claim 18, wherein the digital imaging
pipeline performs image processing and outputs digital still images
that are stored in the storage device.
22. A digital camera comprising: a zoom engine; a display that
displays a plurality of slide show scenarios, each of the slide
show scenarios being associated with a plurality of sequence sets
of motion primitives, one of the motion primitives being a zoom
operation performed by the zoom engine; a button usable to select
one of the slide show scenarios and to select a subset of a plurality
of digital still images stored on the digital camera, the button
also being usable to select audio information stored on the digital
camera; and means for outputting the selected subset of digital
still images and the selected audio information from the digital
camera in accordance with the selected slide show scenario, the
selected subset of digital still images being output as a video
stream, the selected audio information being output as an audio
stream, wherein the video stream includes a sequence of successively
zoomed versions of one of the selected digital still images.
Digital Camera Patent Description
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application claims the benefit under 35 U.S.C. .sctn.119
of Provisional Application No. 60/654,709, entitled "Digital
Camera Having Electronic Video Jockey Capability", filed Feb.
20, 2005 (the subject matter of which is incorporated herein by
reference). This application is related to U.S. patent application
Ser. No. ______, entitled "Electronic Visual Jockey File"
filed Jun. 30, 2005, by Watanabe et al., the contents of which are
incorporated herein by reference.
TECHNICAL FIELD
[0002] The present inventions relate to digital cameras and/or
slide shows involving digital images.
BACKGROUND
[0003] Users of digital cameras often collect a large number of
digital images. This gives rise to a desire to be able to show these
digital images to others in the form of a slide show. Many viewer
programs usable to view digital images provide a slide show feature.
The digital images are typically displayed one at a time, at a constant
rate, in the order in which the digital images are stored in a folder.
There is no audio accompanying the slide show. Consequently, the
slide show is fairly boring to many viewers.
[0004] A product called "PhotoCinema" marketed by a Japanese
company called Digital Stage allows for a fairly sophisticated slide
show to be created and viewed on a computer screen of a personal
computer. Digital images stored on a personal computer can be presented
in a variety of sequences, and individual images in a sequence can
be zoomed. A chain of multiple images can be made to move from left
to right across the computer screen. A chain of multiple images
can be mode to move from top to bottom across the computer screen.
Music can be selected to accompany the slide show. The slide show
is, however, on a computer screen. There is often significant boot
time to start a personal computer, and the computer often does not
have the large screen that would make viewing a slide show an enjoyable
activity. The personal computer may be located in an office or other
out of the way place in the home that does not have the comfortable
seating and lighting of the family room, or media room. Presenting
a slide show on the small screen of a personal computer in the out
of the way room is therefore not as pleasing as it could be.
[0005] Apple Computer has introduced an MP3 music player called
the iPod photo. Some versions of the iPod (called "iPod photo")
have an ability to store a large number of digital images on a built-in
micro hard disc drive. Digital images stored on the iPod can be
viewed in a slide show by coupling the iPod directly to a television.
A special AV (audio/video) cable is provided for this purpose and
the iPod has the ability to drive a video signal and an audio signal
directly to the television. Touch sensitive buttons on the iPod
are usable to select images to be displayed on the television. This
aspect of the iPod is very popular, and the digital images stored
on the iPod can be displayed on a television in the home where comfortable
seating is generally available. It is, however, cumbersome to use
the iPod because the digital images generally need to be loaded
onto the iPod before the iPod can be used to view those images.
This inconvenience and the time required to do the downloading of
images into the iPod is undesirable. Moreover, the slide show generated
by the iPod is fairly simple and constant. There is a constant time-per-slide
value. Watching such a slide show for more than a short period of
time is generally a boring experience.
[0006] Discotheques in the past had disc jockeys (DJs) that played
interesting mixes of music for patrons. There typically was no imagery
or video accompanying the music. The disc jockeys of the past have
been replaced with what are called video or visual jockeys (VJs).
In the dance clubs of today, music is often accompanied by a rich
variety of still images and video clips and light shows and other
imagery and audio and video effects. The VJ may, for example, have
a large expensive stack of many compact disc (CD), digital video
disc (DVD) players, and mixer equipment. The VJ uses this expensive
equipment to combine the output of the various CD players and DVD
players in an interesting fashion to suit the mood of the patrons
of the club. Still images can be seen to sweep across screens in
the club from one side of the screen to another, or from top to
bottom, or from bottom to top as the music is playing. The scene
of view can zoom into a part of an image. The scene of view can
zoom back out from a part of an image. Images can be zoomed up in
size, and can be zoomed down in size. Significant artistry is often
involved in making the collage and flow of pictures and video match
the music so that the overall experience pleasing and has the desired
impact on the audience. Providing this club experience is therefore
generally expensive and requires a significant degree of sophistication.
[0007] It is desired to provide an inexpensive VJ-like experience
for unsophisticated consumers who want to view snapshots in the
home without having to spend a lot of time learning how to program
and use specialized and expensive equipment. It is desired to provide
the VJ-like experience at as low of cost a possible without having
to use a general-purpose computer that is slow to boot and that
may not contain the pictures that are to be viewed. It is desired
to provide the VJ-like experience to users who might not possess
the artistic audio-visual ability of a VJ.
SUMMARY
[0008] A digital camera has a video and audio output ports that
are connectable by cables to an HDTV television. The video cable
may be a YCrCb component video cable. The audio cable may be an
AV cable, the audio portion of which is used to communicate audio
to the television. The digital camera generates a slide show that
is viewable on the television screen. The slide show involves a
sequence of digital still images stored on the camera and audio
stored on the camera. The slide show is supplied to the television
in the form of a video stream and an accompanying audio stream.
[0009] A user uses the digital camera to select the digital images
that will be part of the slide show (a playlist). The user may,
for example, select particular image files (for example, JPEG, BMP,
TIFF, GIF format files) from a list of all the image files stored
on the camera. This list may be displayed on a display of the digital
camera. The user may select files from this list using buttons on
the camera.
[0010] The user also selects one or more audio selections (for
example, MP3, MP4, WAV, AAC, Apple Lossless files, audio snippets
that are captured by the digital camera) that will be part of the
slide show (a playlist). The user may, for example, select a particular
audio selection from a list of audio selections that are stored
on the digital camera. This list is displayed on the display of
the digital camera. The user identifies an audio selection from
the list using buttons on the camera.
[0011] The digital camera has both a wireless transceiver port
(for example, FIR IRDA or BlueTooth or UWB) as well as a port for
accommodating a cable or docking station (for example, USB 2.0).
These ports are usable to download images and/or audio for including
into the slide show. The images and/or audio can be downloaded from
any suitable repository of image and audio information (for example,
a personal computer, an MP3 player, another digital camera, a cell
phone, or a personal digital assistant). The information can also
be ported to the digital camera using a removable storage media
(for example, a removable flash card, a memory stick, a removable
hard disc drive, or an optical disc).
[0012] Using the display and push buttons on the camera, the user
selects one of a plurality of "scenarios" for the slide
show. The particular scenario selected determines how the selected
digital images and the selected audio will be presented in the slide
show. A scenario may involve multiple "sequence sets,"
where a sequence set is a predefined specification of how images
will be manipulated in an artistic VJ-like fashion (manipulations
include blending, panning, tilting, zooming, rotating). A sequence
set can also control aspects of the audio such as fade in, fade
out, volume, and changing the audio file being decoded and output.
In one novel business method aspect, an experienced visual jockey
is consulted to develop elements, sequence sets and scenarios that
have high artistic quality. These elements, sequence sets and scenarios
are then provided in a production version of the digital camera
for use by ordinary consumers.
[0013] A powerful hardware zoom engine that performs sub-pixel
zooming and that is in used in the digital camera to capture digital
images is also used during the generation of the slide show to perform
operations such as zooming, panning, and tilting operations. The
digital camera electronics including the powerful hardware zoom
engine performs these operations in real time as the slide show
progresses. Unlike a situation where an iPod is used to generate
a slide show, the powerful hardware is provided in the camera for
image capturing purposes and providing a sophisticate zoom engine
does not entail added cost for the consumer.
[0014] In one embodiment, the user can stop and start the slide
show using a button on the digital camera. The user can also use
buttons on the digital camera to cause a pointer to appear on the
television screen and to move the pointer around the television
screen.
[0015] In some embodiments, the user can customize a scenario in
certain ways. Once customized, the customized scenario is used in
a subsequent slide show. Automatic face detection within the camera
is employed to make the slide show more interesting and VJ-like.
The location of a face can, for example, be used to control which
parts of an image are emphasized. Face detection can also be used
to determine which one of a plurality of images will be emphasized
over the others. The detection of a face is provided as an input
to the slide show generating software.
[0016] In some embodiments, the beat of the audio that accompanies
the sequence of images in the slide show is detected. The beat is
provided as an input to the slide show generating software. The
beat is used to synchronize and time the sequencing of digital images
to the accompanying audio to make the slide show more interesting
and VJ-like.
[0017] In another novel aspect, a single container file contains
content files as well as textual information on how to render content
in the content files so as to render a slide show in accordance
with a scenario. The textual information may, for example, be present
in the form of a text file that is contained in the container file.
The content files may, for example, include JPEG image files and
MP3 audio files. In addition, the single container file may include
a textual playlist file that identifies file names of content files
that are to be rendered during the slide show. The container file
adheres to format requirements for a new standard type of file.
This type of file may be called an EVJ file. EVJ stands for electronic
visual jockey. EVJ files names may end with .EVJ to denote that
they are EVJ files.
[0018] A rendering device that includes an EVJ rendering/authoring
functionality and that comports with the EVJ standard can be used
to read the EVJ file, to parse the text information, and to render
content in the content files so as to regenerate the slide show
in accordance with the originally specified scenario. The regenerated
slide show appears substantially the same as the slide show was
originally authored by the slide show creator using another rendering
device. A rendering device may, for example, be a digital camera,
a desktop personal computer, a laptop personal computer, a television,
a combination of a cable set-top box and a display device, a combination
of satellite set-top box and a display device, a combination of
a digital video disc (DVD) player and a display device, a hand-held
slide-show viewing device, a combination of a hand-held slide-show
viewing device and a display device, a cellular telephone, an MP3
player, a personal digital assistant (PDA), a combination of a home
entertainment center control unit and a television.
[0019] A user can use a digital camera having an EVJ rendering/authoring
functionality to select a plurality of content files and a scenario
for a slide show. The rendering device then generates an EVJ file
in the proper EVJ format. To view the slide show on a rendering
device, the EVJ rendering/authoring functionality accesses the EVJ
file, and parses the text in the EVJ file, and from the text generates
a sequence of content manipulation instructions. The content manipulation
instructions are carried out by the rendering device such that the
content is rendered so as to play the slide show again. Such an
EVJ file can be communicated or transferred (for example, emailed
or transferred by flash memory card) from the creator of the EVJ
file to a second person. The second person can then use a second
rendering device that has an EVJ rendering/authoring functionality
to render the slide show on the second rendering device in the same
way that the EVJ file was rendered by the creator on the first rendering
device. Functionality in the rendering device allows an EVJ file
to be edited so that when the modified EVJ file is rendered, the
slide show is seen in its altered form. Examples of editing that
can be performed include adding images, deleting images, changing
the order that images are rendered, adding text that will be displayed
during the slide show, adding audio snippets, deleting audio snippets,
changing the music that accompanies the imagery of the slide show,
and editing the definition of the scenario in the EVJ file.
[0020] Other embodiments and advantages are described in the detailed
description below. This summary does not purport to define the invention.
The invention is defined by the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings, where like numerals indicate
like components, illustrate embodiments of the invention.
[0022] FIG. 1 is a perspective view of a slide show system in accordance
with one embodiment.
[0023] FIG. 2 is a more detailed view of the back side of the digital
camera of the slide show system of FIG. 1.
[0024] FIG. 3 is a simplified diagram of the electronic circuitry
of the digital camera of FIG. 1. An imamge processing integrated
circuit 100 as illustrated can be realized as part number SiP-1280
Digital Processor available from NuCORE Technology, Inc. of 1380
Bordeaux Drive, Sunnyvale, Calif.
[0025] FIG. 4 illustrates the hierarchical structure of slide show
generating software that executes on the processor within the image
processing integrated circuit of the digital camera of FIG. 1. The
processor is the same processor that controls the camera during
the capturing of digital images.
[0026] FIG. 5 is a simplified diagram that illustrates seven sequence
sets. Each sequence set is made up of multiple elements. The sequence
sets and elements may, for example, be developed with the assistance
of an artist, film maker, or VJ.
[0027] FIG. 6 is a diagram illustrative of the actions to be performed
to carry out the element 45FR_02.
[0028] FIG. 7 is a diagram illustrative of the actions to be performed
to carry out the element 45FR_04.
[0029] FIG. 8 is a diagram illustrative of the actions to be performed
to carry out the element 15FR_02.
[0030] FIG. 9 is a diagram illustrative of the actions to be performed
to carry out the element FB90FR_10.
[0031] FIG. 10 illustrates a menu of scenario customization options
that appear on the display of the camera of FIG. 1.
[0032] FIG. 11 is a simplified diagram of an EVJ file.
[0033] FIG. 12 is an example of an environment in which the EVJ
file of FIG. 11 can be used.
[0034] FIG. 13 is a simplified diagram of method involved in the
rendering of the EVJ file of FIG. 11.
DETAILED DESCRIPTION
[0035] Reference will now be made in detail to some embodiments
of the invention, examples of which are illustrated in the accompanying
drawings.
[0036] FIG. 1 is a perspective view of a slide show system 1 in
accordance with one embodiment. In an initial step (not shown),
a user 2 uses a digital still camera 3 to capture digital still
images and to store the digital still images in the form of files
(for example, JPEG files) on digital camera 3. Digital image files
can also be loaded onto digital camera 3 from another repository
of digital image files (for example, a personal computer, a PDA,
a cell phone, another digital camera, a portable micro hard disc
drive, a portable memory, optical disc). Digital camera 3 includes
a microphone that is capable of capturing snippets of audio information
(for example, audio present at the time a digital still image is
captured by the digital camera). These snippets of audio information
are also stored as discrete files on digital camera 3. Music in
digital form (for example, MP3, WAV, MP4, AAC, Apple Lossless format
files) can also be loaded onto the digital camera from a repository
of digital information (for example, a personal computer, a CD player,
a DVD player, an MP3 player, a PDA, a cell phone, portable micro
hard disc drive, portable memory, optical disc).
[0037] Once the digital still images and the audio information
is present on digital camera 3, user 2 uses digital camera 3 to
display the digital still images and audio information to another
individual 4 in an interesting, VJ-like slide show. The showing
of the VJ-like slide show is illustrated in FIG. 1. To present the
slide show, digital camera 3 is coupled by cables 5 and 21 directly
to an HDTV (high definition television, 1280.times.720) television
6. Cable 5 is a YCRCB component video cable used to communicate
video to television 6. Cable 21 is an AV composite cable 21, the
audio part of which is used to communicate audio to television 6.
Digital camera 3 outputs the digital still images in a sequence
in the form of a stream of video information across cable 5 to HDTV
television 6 such that the slide show is seen on the television
screen. Digital camera 3 outputs the accompanying audio (for example,
music and/or audio snippets) across cable 21 to television and the
audio is heard on the speakers of the television 6 or other sound
system used with the television. User 2 controls the slide show
using a display 7 of the digital camera and using buttons 8-14 on
the digital camera.
[0038] Most if not all of the hardware necessary to provide digital
camera 3 the ability to output the video stream and audio stream
is already present in conventional digital cameras (for example,
digital still cameras employing image processing integrated circuits
manufactured by NuCORE Technology Inc. of Sunnyvale Calif.). In
the near future, many consumers will have in their homes both digital
cameras of this type as well as HDTV televisions. Consequently,
the principal added hardware cost associated with providing user
2 the added capability of being able to show individual 4 the VJ-like
slide show is cables 5 and 21. Where digital camera 3 is of a type
that includes a large amount of storage space (for example, due
to the camera including a micro hard disc drive), a tremendous number
of high resolution digital still images can be stored on the digital
camera, thereby obviating a need on the part of a consumer to purchase
an additional expensive device such as an iPod just to store and
to display digital still photographs. There is no need to capture
digital still images with a first device (a camera) and then load
the captured digital still images onto a second device (an iPod)
that has adequate storage to store all the consumer's digital pictures.
Digital camera 3 performs both the image capturing function of a
camera as well as the digital image storing and displaying functions
of an iPod.
[0039] FIG. 2 is a more detailed view of the back side of digital
camera 3. The back side includes display 7 (for example, an LCD
display), four directional buttons 8-11, a pointer navigating nipple
12, a MENU push button 13, an ENTER push button 14, and a view finder
15. Digital camera 3 also includes a video output port 16. Video
output port 16 is shown directly coupled to HDTV television 6 via
a video cable 5. Digital camera 3 also includes an audio output
port 20. Audio output port 20 is shown directly coupled to HDTV
television 6 via an audio cable 21. Digital camera 3 also includes
a wireless transceiver port 17 (for example, including an ultra
wide band (UWB) transceiver, an FIR IRDA (infrared data association)
transceiver, or a Bluetooth transceiver). Wireless transceiver port
17 is usable, for example, to receive digital still images from
another wireless device, or to receive digital audio information
from another wireless device so that the received images and/or
audio information can be incorporated into a VJ-like slide show.
Digital camera 3 also includes a port 18 that accommodates a physical
connector (for example, a USB 2.0 port). Port 18 is shown coupled
to a personal computer 19 via a USB cable. Digital camera 3 can
receive digital still images and/or digital audio information from
personal computer 19 so that these images and/or audio information
can be incorporated into a VJ-like slide show.
[0040] FIG. 2 also shows what is displayed on display 7 when user
2 uses digital camera 3 to set up the VJ-like slide show. A plurality
of digital image file indicators, a plurality of digital audio file
indicators, and a plurality of slide show scenario indicators are
displayed. In the example of FIG. 2, the digital image file indicators
are file names (for example, PHOTO#1.JPG) shown in a column. In
the example of FIG. 2, the digital audio file indicators are file
names (for example, SONG#1.MP3) shown in a column. In the example
of FIG. 2, the slide show scenario indicators are text names (for
example, SCENARIO#1) shown in a column. Each digital image file
indicator represents an image file stored on digital camera 3 and
each digital audio file indicator represents an audio file stored
on digital camera 3.
[0041] User 2 uses the four directional buttons 8-11 to identify
one of the indicators. A currently identified indicator appears
highlighted. The four directional buttons 8-11 are usable to move
the highlighting from one indicator to the next, up and down a list
of indicators, and left and right across the columns of indicators.
When user 2 has highlighted a desired indicator, user 2 presses
ENTER button 14 to select the indicator. Once selected, the indicator
remains highlighted even if the directional buttons 8-11 are used
to move the identified indicator away to another indicator. In this
way, user 2 selects a plurality of indicators for digital still
images stored on digital camera 3. In the illustration, three digital
still image indicators are selected, PHOTO#2.JPG, PHOTO#4.JPG and
PHOTO#5.JPG. The list of digital still image indicators can scroll
so that a larger number of indicators than seven is available for
selection even though only seven can be displayed at a time.
[0042] In the same way that user 2 selects a plurality of digital
still image indicators, the user 2 uses directional buttons 8-11
and ENTER button 14 to select one of more digital audio indicators
(in the present case, the audio information selected is a song SONG#3
stored in MP3 format on the digital camera).
[0043] User 2 also selects one of the listed slide show scenario
indicators. One scenario may be suited for use with easy listening
contemporary music such as smooth jazz. Another scenario may be
more suited for use with classic symphonic music. Another scenario
may be more suited for use with dance and club music with a rapid
beat. Another scenario may be more suited for use with high-energy
rock music.
[0044] In the present example, SCENARIO#6 is selected. During this
selection process, the information displayed on display 7 may also
be displayed on the screen of HDTV television 6. Digital camera
2 includes on-screen display circuitry and an image-blending capability
usable for the purpose of displaying whatever is shown on display
7 on the screen of HDTV television 7.
[0045] FIG. 3 is a simplified diagram of the electronic circuitry
of digital camera 3. Digital camera 3 includes optics 100, an image
sensor 101, an analog front end and timing generator integrated
circuit 102, an image processing integrated circuit 103, an amount
of SDRAM 104 for the buffering of image information, display 7,
a microphone 105, a speaker 106, video output port 16, audio output
port 20, wireless transceiver port 17, USB bus port 18, an amount
of mass storage (for example, removable flash memory and/or a micro
hard disc drive) 107, push buttons including buttons 8-14, and a
microcontroller 108. Microcontroller 108 performs, among other functions,
key-scanning (button-scanning) functions to detect the pressing
of a button. Microcontroller 108 communicates with image processing
integrated circuit 103 across a serial bus and employs serial interface
circuitry 109. Image processing integrated circuit 103 includes,
among other parts, a digital imaging pipeline 110, a processor 111,
a zoom engine 112, an HDTV NTSC/PAL codec 113, and an audio DAC/ADC
(an digital-to-analog converter and an analog-to-digital converter)
115. On screen display (OSD) circuitry is provided in the interface
circuitry 114 for the display so that text and icons can be displayed
over a background image.
[0046] FIG. 4 illustrates the structure of slide show generating
software 200 that executes on processor 111 within image processing
integrated circuit 103. The bottom row of blocks in the diagram
represents discrete blocks of hardware circuitry. Each block of
hardware circuitry is controlled by software using the hardware
object (hardware driver) layer 201. The layer 202 entitled "motion
primitive engine" presents a plurality of motion primitive
APIs (application programming interfaces) that are made available
to higher level software. An example of a motion primitive API is
an API call to zoom into a particular location within a particular
JPEG image. Such a motion primitive API may be used multiple times
to successively zoom into a particular location within a desired
digital still image.
[0047] Another example of a motion primitive API is an API call
to zoom up the size (increase the size as displayed) of a digital
image. Another example of a motion primitive API is an API call
to zoom down the size (decrease the size as displayed) of a digital
image.
[0048] Another example of a motion primitive API is an API call
to move a particular image to the left with respect to a background
image. This operation is called "panning." The panning
API may be called multiple times in succession to move a particular
image to the left across a background image. There is another similar
API for panning to the right with respect to a background image.
The background image can be another captured image, or alternatively
can be a solid white frame or a solid black frame.
[0049] Another example of a motion primitive API is an API call
to move a particular image upward with respect to a background image.
This operation is called "tilting." Such a tilting motion
primitive API may be used multiple times to successively move a
particular image upward across a background image. There is another
similar API for tilting downward with respect to a background image.
The background image can be another captured image, or alternatively
can be a solid white frame or a solid black frame.
[0050] Another example of a motion primitive API is an API call
to perform what is called "filtering" on a particular
image. In this sense, filtering involves blending with a blurring
effect.
[0051] Although the layer of software is called the motion primitive
API layer, there are APIs presented by this layer that are not for
motion primitives. There are, for example, APIs for controlling
the audio DAC/ADC and the HDTV video codec. For example, one API
can be called to start the decoding of audio information of a particular
file. Codec software executing on processor 111 performs the decoding
function and outputs the result to the audio DAC which in turn outputs
analog audio onto audio out port 20. Another API can be called to
cause digital still image information from a particular file to
be supplied to the HDTV video codec such that the codec outputs
that information in the form of a video stream onto video out port
16.
[0052] There is an API for blending a first image with a second
image. A variable field in the API call indicates the proportion
(in a percentage number) of the resulting image that is to be of
the first image. If for example the proportion is set at thirty
percent, then each pixel value of the first image will be multiplied
by thirty percent and each corresponding pixel value of the second
image will be multiplied by seventy percent, and the two products
will be summed to arrive at the pixel value for the resulting image.
In the present embodiment, this blending function is performed by
processor 111.
[0053] The next layer of software above the motion primitive engine
layer 202 is the "sequence set API layer" 203. FIG. 5
is a simplified diagram that illustrates seven sequence sets. Each
sequence set is a sequence of elements. Sequence set SET_06, for
example, defines the sequence of elements: 45FR_02 (an element 45
frames in duration); 45_FR_04 (an element 45 frames in duration);
15FR_02 (an element 15 frames in duration); and FB90FR_10 (an element
90 frames in duration). Sequence set SET_06 therefore defines a
total sequence of 195 frames. Each element defines how an associated
set of digital images will be zoomed, panned, tilted, rotated, and
blended. A sequence set and what it entails in one embodiment is
described in further detail below.
[0054] The next layer (see FIG. 4) of software above the sequence
set layer 203 is the scenario layer 204. In the illustrated example,
there are N possible slide show scenarios from which a user can
choose one. Each of these scenarios identifies a sequence of particular
sequence sets. When one sequence set associated with a scenario
is completed, then the next sequence set associated with the scenario
is started, and so on. A scenario and what it entails is described
in further detail below.
[0055] The next layer of software above the scenario layer 204
is the scenario controller layer 205. This layer defines the particular
digital images that are to be associated with the selected scenario.
The layer also defines the particular audio information that is
to be associated with the selected scenario.
[0056] The top layer of software above the scenario controller
layer 205 is the user interface (UI) software layer 206. UI layer
software controls what is displayed on display 7 during the set
up of the slide show. UI layer software detects which buttons are
pressed and takes appropriate action to change what is displayed
on display 7 and to set up the slide show.
[0057] Continuing on with the example of user 2 selecting SCENARIO#6
in FIG. 2, the digital camera 3 determines, due to SCENARIO#6 being
selected, that sequence sets SET_06, SET_02 and SET_01 are to be
rendered and in that order. From the first sequence set SET_06 (see
FIG. 5), the first element is element 45FR_02.
[0058] FIG. 6 is a diagram illustrative of the actions to be performed
to carry out this element 45FR_02. Two digital images are associated
with this element: PHOTOA and BLACK. The digital images selected
by user 2 are taken in top down order. The uppermost selected digital
image PHOTO#2.JPG (from the list as displayed on display 7) is therefore
assigned to be PHOTOA. The digital image BLACK is not an actual
digital image stored in memory, but rather is a solid field of black
pixel information. In FIG. 6, time extends in the direction from
left to right. The numbers 0F, 5F, 10F and so forth across the upper
part of the diagram represent the number of the frame of video.
The "ZOOM RATIO" value 100(0FR) indicates that PHOTOA
is at full size (i.e., zoom=100%) at the time of frame zero. The
"ROTATION" value of 0(0FR) indicates that PHOTOA is rotated
by zero degrees (i.e., there is no rotation of PHOTOA). The "BLENDING"
value 0(0FR) indicates that PHOTOA is at zero percent strength versus
the other digital image (the BLACK digital image) at the time of
frame zero. Accordingly, the result of the blending is an entirely
black digital image.
[0059] At time 10F, the zoom ratio and rotation values remain the
same, so PHOTOA is not zoomed up or down in size and is not rotated.
The blending ratio has, however, gone from a value of zero percent
at the time of frame zero to a value of one hundred percent at the
time of frame ten. This indicates that at the time of frame ten,
PHOTOA is blended with a factor of one hundred percent and the black
image is blended with a factor of zero percent. Accordingly, the
black image at time zero is gradually blended away with PHOTOA until
PHOTOA appears at time 10F unaffected by the black image.
[0060] At time 20F, the zoom ratio is 115(20FR). It is therefore
seen that PHOTOA is zoomed up in size so that PHOTOA appears 115
percent of its starting size at time 20F. Note that the same high
performance zoom engine is used both to capture images as well as
to perform zooming functions during a slide show.
[0061] Nothing changes from the time of frame 20F until time of
frame 30F. At time 35F, the zoom ratio is 40(35FR). PHOTOA is therefore
zoomed down, starting at time 30F and ending at time 35F, to be
forty percent of its starting size.
[0062] At time 40F, the tilting of PHOTOA upward is begun as indicated
by the left block labeled 40FR that appears underneath the time
line. Note the starting position of the forty percent size version
of PHOTOA is centered in the screen. The upward pointing arrow indicates
the upward tilting of PHOTOA.
[0063] At time 44F, the upward tilting of the forty percent size
version of PHOTOA has reached the location indicated by the block
below the time line labeled 44FR. Note that PHOTOA remains at forty
percent of its ordinary size. Element 45FR_02 ends at the time of
frame 45.
[0064] The zooming, rotation, blending, and tilting indicated by
the sequence set of FIG. 6 are motion primitives, each of which
is performed by calling a corresponding motion primitive API. Although
not shown here, an element can also include API calls to control
how the audio is output. An element can cause a certain audio file
to start to be decoded and output to the television. An element
can cause a volume level to be changed. An element can cause fade
in and fade out audio. An element can stop the decoding of a particular
audio file.
[0065] FIG. 7 illustrates the next element (element 45FR_04) in
sequence set SET_06 of scenario SCENARIO#6. Two digital images are
associated with this element: PHOTOA and WHITE. As in the case of
the BLACK frame of element 45FR_02, the WHITE frame is not an actual
digital image stored in memory, but rather is a solid field of white
pixel information.
[0066] At the time of frame 0F, the second digital image in the
digital still images selected by user 2 (in this example, the digital
still image represented by PHOTO#4.JPG) is PHOTOA. As indicated
by the 40(0FR) appearing in the line labeled "ZOOM RATIO,"
PHOTOA is zoomed to be forty percent of its original size. The starting
position of PHOTOA is illustrated in the leftmost block labeled
0FR appearing below the timeline. PHOTOA starts out of the field
of view, and tilting starts upward with respect to the background
frame.
[0067] At the time of frame 5F, the forty percent size version
of PHOTOA has reached the center of the field of view as indicated
by the rightmost block labeled 5FR appearing below the timeline.
[0068] From the time of frame 5F to the time of frame 10F, there
is no change. At the time of frame 15F, the zoom ratio is one hundred
percent. Accordingly, starting at time 10F, PHOTOA is zoomed up
in size so that it is one hundred percent of its original size by
time 15F.
[0069] From the time of frame 15F to the time of frame 25F, there
is no change. Between the times of frames 25F and 45F, the blending
percentage changed from one hundred percent at time 25F to zero
percent at time 44F. The one hundred percent sized version of PHOTOA
is therefore seen to be blended away into a solid white background
by time 44F. The element stops at time 45F.
[0070] FIG. 8 illustrates the next element (element 15FR_02) in
sequence set SET_06 of scenario SCENARIO#6. There are three frames
involved in this element, a solid white image frame, PHOTOA (which
is the next digital still image selected by user 2), and a solid
white image frame. PHOTOA in this case is the digital still image
represented by PHOTO#5.JPG.
[0071] Initially, at the time of frame 0F, the solid white frame
is rendered at one hundred percent size but it is blended with a
percentage of zero percent. PHOTOA, on the other hand, is blended
with the solid black frame with a percentage of zero percent. The
overall result is the solid black frame output at the time of frame
0F.
[0072] The blending percentage of PHOTOA versus the solid black
frame, however, changes from zero percent at time 0F to one hundred
percent at time 7F. PHOTOA is therefore seen to emerge from a black
screen and appears in undarkened form by time 7F.
[0073] The blending percentage for the solid white frame remains
at zero percent from the time of frame 0F to the time of frame 8F.
From time 8F to time 11F, the blending of the solid white frame
changes from zero percent to one hundred percent. PHOTOA at time
0F is at a zoom ratio of zero as indicated by 100(0FR) and at time
11F is at a zoom ratio of 107 as indicated by 107(11FR). PHOTOA
therefore is at size 107 at the time of frame 11F, but it is blended
with a solid white frame at time 11F so the result is a solid white
frame.
[0074] From time 11F to time 14F, the blending ratio of the solid
white frame goes from one hundred percent to zero percent, and the
blending of PHOTOA goes from one hundred percent blending with the
black frame at time 11F to zero percent at time 14F. The result
is a gradual shift from an all white frame at time 11F to an all
black frame at time 14F.
[0075] FIG. 9 illustrates the next element (element FB90FR_10)
in sequence set SET_06 of scenario SCENARIO#6. There are six frames
involved in this element, designated PHOTOA, PHOTOB, PHOTOC, PHOTOD,
PHOTOE and PHOTOF in FIG. 9. PHOTOA in this case is a digital still
image selected by the user using the user interface seen in FIG.
2 but the selected digital still image file name is not one of the
file names highlighted in FIG. 2 but it is rather in the part of
the list of image files that is not displayed on the screen.
[0076] At the time of frame 0F, the digital still images appear
in the positions indicated in the left diagram in the leftmost block
below the time line of FIG. 9. The arrow illustrates the direction
of panning.
[0077] At the time of frame 10F, the digital still images appear
in the positions indicated in the right diagram image in the rightmost
block below the time line of FIG. 9. The digital still image PHOTOF
has moved out of the field of view, and PHOTOD has moved into the
field of view. Each of the digital still images PHOTOA-PHOTOF are
of at thirty percent size as indicated by the time line. The panning
stops at the time of frame 10F. This can be indicated by a stop
panning indicator (not shown) in the position rows of the time lines
of PHOTOA-PHOTOF.
[0078] Starting at the time of frame 15F, only one of the digital
still images, PHOTOB, that is in the middle of the field of view,
is increased from the thirty percent size to seventy percent size
at the time of frame 19F, remains at seventy percent size until
the time of frame 30F, and then is decreased in size to thirty percent
size at the time of frame 34F. This emphasizes PHOTOB.
[0079] This condition remains until the time of frame 40F, when
the panning resumes. The resumption of panning can be indicated
by a start panning indicator (not shown) in the position rows of
the time lines of PHOTOA-PHOTOF. The row of digital still images
PHOTOA-PHOTOF is seen to start to pan to the right across the field
of view at time 40F. The digital still images PHOTOA-PHOTOF appear
in the positions indicated in the left diagram in the rightmost
block below the time line of FIG. 9. The arrow illustrates the direction
of panning. The panning stops at the time of frame 50FR. At the
time of frame 50FR, the digital still images PHOTOA-PHOTOF appear
in the positions indicated in the right diagram in the rightmost
block below the time line of FIG. 9. From the time of frame 50F
to the time of frame 90F, the positions of the images remains fixed.
[0080] After sequence set SET_06 has been rendered, the next sequence
set of SCENARIO#6, sequence set SET_02 is rendered. Then the last
sequence set of the scenario, sequence set SET_01 is rendered. This
completes the slide show.
[0081] At any time in the slide show, the user can halt the slide
show by pressing the ENTER button 14. Pressing the ENTER button
14 again restarts the slide show from the point in the scenario
where the slide show was halted. The ENTER button 14 acts as a toggle
button for this stop/restart feature.
[0082] Digital camera 3 has a pointer overlay that can be superimposed
over whatever images are being displayed on television screen 6
during the slide show. The pointer overlay can be made to appear
when the slide show is in progress, and can also be made to appear
when the slide show is stopped due to use of the ENTER button 14
as set forth above. To cause the pointer overlay to appear, the
user manipulates one of the directional buttons 8-11 or manipulates
the pointer navigating nipple 12. Microcontroller 108 detects movement
of one of these buttons, and informs the image processing integrated
circuit 103 via the serial bus and serial interface circuitry 109.
The processor 111 within the image processing integrated circuit
103 in conjunction with codec 113 superimposes the pointer on the
digital image being displayed on television screen 6 so that the
pointer appears as an overlay. Accordingly, pointer information
is embedded in the video stream sent to television 6 so that the
pointer will appear on the screen of television 6. The user can
move the pointer around the television screen of television 6 by
manipulating the directional buttons and/or the pointer navigating
nipple. The pointer can be made to disappear by having the user
take an appropriate action, for example by pressing the ENTER button
14 when the pointer is being displayed. In this case, pressing the
ENTER button 14 does not stop or start the slide show, but rather
causes the pointer to disappear from the screen of the television.
[0083] The slide show as orchestrated by the selected scenario
can, in some embodiments, be customized. In one embodiment, user
2 can stop the slide show at a point where customization is to occur
by pressing the ENTER button 14. Once the slide show is stopped,
user 2 presses the MENU button 13, thereby causing a menu of customization
options to appear.
[0084] FIG. 10 illustrates an example of customization options
that can be made to appear on display 7. If a location in a digital
still image is being zoomed into (is the center of the zoom field)
at the time the slide show was halted, the user can change the location.
Consider for example a situation in which a scene includes an object
of interest and a background. If the scenario is zooming into an
uninteresting point in the background, the user can change the point
being zoomed into to be a point on the object of interest. Likewise,
if the action is a zoom out, and if the point in the digital image
being used as the center of the zoom out is not an object of interest
but rather is an uninteresting point in the background, then the
user can change the point to be a point on the object of interest.
[0085] Another option on the menu of FIG. 10 is to move the current
digital image one image later in the sequence of images. To do this,
the directional buttons are pressed to move the highlighted menu
option onto the "MOVE PICTURE LATER" option. Pressing
the ENTER button performs the operation.
[0086] Another option is to move the current digital image one
image earlier in the sequence of images. To do this, the directional
buttons are pressed to move the highlighted menu option onto the
"MOVE PICTURE EARLIER" option. Pressing the ENTER button
performs the operation.
[0087] Another option is to capture an audio clip. To do this,
the directional buttons are pressed to move the highlighted menu
option onto the "CAPTURE AUDIO CLIP" option. Pressing
the ENTER button causes the digital camera to start capturing audio
from microphone 105. The capturing may, for example, stop after
a certain amount of time or when the sound being recorded drops
below a certain loudness threshold for a predetermined amount of
time. The last captured audio clip can be added into the slide show
starting at the time when the slide show was halted by selecting
the ADD AUDIO CLIP option and pressing the ENTER button. The added
audio clip is output to the television rather than the selected
SONG#3.MP3 until the audio clip reaches its end in which case the
playing of the SONG#3.MP3 resumes. If an audio clip is being played
at the time when the slide show was halted, the audio clip can be
deleted from the slide show by selecting the "DELETE AUDIO
CLIP" and pressing the ENTER button.
[0088] The volume of the accompanying audio can be changed by stopping
the slide show, selecting "VOLUME UP" or "VOLUME
DOWN" on the menu of FIG. 10, and then pressing the ENTER button.
[0089] The customization of the scenario is stored in the digital
camera. To replay the slide show with the scenario as modified,
the user presses the MENU button 13 which toggles the menu displayed
to the menu of FIG. 2. The user then presses the ENTER button without
making any PICTURE, MUSIC or SCENARIO changes. This causes the last-selected
scenario, SCENARIO#6, to be replayed. When the scenario is replayed,
it is replayed as customized by the user. The menu options of FIG.
10 are merely some of the possible ways that a slide show scenario
can be changed by a user or customized. The options of FIG. 10 are
presented as examples. Other ways of customizing a slide show can
be listed on the menu of FIG. 10.
[0090] In one embodiment, the digital camera has the ability to
detect a human face in a digital still image, and to use the location
of the detected face to control how the digital image is zoomed,
panned, tilted or otherwise manipulated in the slide show. The existence
of a face in an image can, for example, be detected by processing
the image to identify areas of image pixels that are in a skin tone
color range. The bottom edges of these areas are then determined.
The bottom edges are then compared with an arcuate, U-shaped, template
of a human chin to see if one of the bottom edges matches the template.
A correlation value between the bottom edge and the template is
determined. If the correlation value is over a threshold, then the
skin tone region is judged to be a face. For additional details
on how face detection can be performed, see: 1) U.S. patent application
Ser. No. 10/970,804, entitled "Template Matching Method And
Target Image Area Extraction Apparatus", filed Oct. 21, 2004
(the content of which is incorporated herein by reference), 2) the
article entitled "A Real-Time Multi Face Detection Technique
Using Positive-Negative Lines-of-Face Template", by Yuichi
Hori et al., Proceedings of the International Conference of Pattern
Recognition (ICPR '04), vol. 1, no. 1, pages 765-768 (2004), (the
content of which is found in provisional patent application 60/654,709
that is incorporated herein by reference), and 3) the slides entitled
"Using Genetic Algorithm As An Application of Wireless Interconnect
Technologies for LSI Chip," by Yuichi Hori, (the content of
which is found in provisional patent application 60/654,709 that
is incorporated herein by reference).
[0091] If, for example, the scenario of the slide show defines
a zooming into a particular location of an image, and if the image
is determined to include a human face as described above, then the
location being zoomed into is automatically set to be the location
of the detected face. Similarly, if the scenario of the slide show
defines a zooming out from a particular location of an image, and
if the image is determined to include a human face as described
above, then the location from which the zooming out occurs is automatically
set to be the location of the detected face. If another motion primitive
is to be performed and that motion primitive can be adjusted to
emphasize one part of an image over another, then the location where
the face was detected can be used to emphasize the part of the image
containing the human face.
[0092] In one embodiment, the audio information selected in FIG.
2 for accompanying of the slide show is preprocessed to detect a
tempo of the audio information. The envelope of the audio loudness
can, for example, be analyzed in the frequency domain so that a
dominant frequency such as, for example, a drum beat is detected.
The timing of this beat of the audio is stored with the audio information.
When the scenario is selected and the slide show is started, the
result of the previous audio analysis is used to control the rate
of change of the images in the slide show. In one example, the appearance
and changing of digital images is timed to coincide with the detected
audio beat.
[0093] The features described above combine to make the slide show
generated by digital camera 3 of FIG. 1 more "VJ-like,"
less repetitive, and less boring than a conventional slide show
as generated by an iPod. By providing the slide show generating
capability in the digital camera used to capture the images, a consumer
need not purchase an additional costly device such as an iPod just
to be able to generate a slide show that is viewable on the television.
Where digital camera 3 has a large storage capacity, the often onerous
task of having to transfer captured digital images from a camera
or personal computer to another device (such as an iPod) that has
more storage capacity is reduced or eliminated. Digital camera 3
can both store a large collection of digital images as well as generate
interesting VJ-like slide shows that can be viewed on a television
without involving additional electronic devices.
[0094] In one alternative embodiment, the structure of FIG. 4 is
modified to include a motion primitive script layer. The motion
primitive script layer appears above the dark solid horizontal line
in FIG. 4 and below the sequence set layer 203. Motion primitive
engine 202 is not provided, but rather box 202 is a motion primitive
interpreter. The motion primitive interpreter interprets a motion
primitive script and as a result instructs the lower level hardware
layer how to carry out appropriate motion primitives specified in
the script.
Single File Containing Image Files and Textual Slide Show Definition
[0095] FIG. 11 is a diagram of a single container file 300 that
defines a slide show and contains the content to be rendered in
the slide show. Container file 300 may, for example, be called an
"EVJ file" where "EVJ" stands for electronic
visual jockey. Container file 300 may, for example, have a file
name that ends in ".EVJ".
[0096] Container file 300 contains a plurality of content files,
and an amount of text that defines a slide show scenario. In the
illustrated example, there are six content files 301-306. Content
files 301-304 are JPEG files of still image information. Content
files 305 and 306 are .MP3 files containing digital music information.
The text that defines the slide show is present in the form of a
text file 307. Text file 307 is referred to here as a scenario file.
In addition to text file 307 and the content files 301-306, container
file 300 may also include another text file 308 called a playlist
file. The playlist file 308 contains a list of file names of content
files, where the content in the content files will be rendered during
the slide show.
[0097] FIG. 12 is a diagram of one example of an environment in
which EVJ container file 300 may be used. In one example, digital
camera 3 is used by a user select a scenario and a plurality of
content files. Digital camera 3 then renders a slide show in accordance
with the selections and at the same time generates the EVJ container
file 300. EVJ container file 300 is stored on a removable flash
memory card 309 that is coupled to digital camera 3. The user can
then remove flash memory card 309 from digital camera 3 and plug
memory card 309 into an accommodating slot 310 in a high definition
television (HDTV) 311. HDTV 311 includes an EVJ rendering/authoring
functionality 312 as well as standard television electronics 313.
A user can control the television using a remote control device
314 in ordinary fashion.
[0098] In the present example, after the user plugs memory card
309 into slot 310 in HDTV 311, the user uses remote control device
314 to initiate the EVJ rendering/authoring functionality 312 in
accessing and reading the EVJ file 300.
[0099] FIG. 13 is a simplified flow chart of a method of using
EVJ file 300 to render a slide show. The step of the EVJ rendering/authoring
functionality 312 accessing the EVJ file 300 is step 400 in FIG.
13.
[0100] EVJ rendering/authoring functionality 312 in HDTV 311 begins
parsing the text in the scenario file 307. In the illustrated example,
scenario file 307 defines the scenario that was previously selected
by the user of the digital camera 3, where the scenario is a sequence
of sequence sets, and where each sequence set is defined as a sequence
of elements. EVJ rendering/authoring functionality 312 reads the
text of the textual scenario file 307 in order (top-down order in
FIG. 11). This parsing of a first bit of the text in the textual
scenario file 307 is step 401 in FIG. 13.
[0101] A code 315 for an element is found in the text scenario
file 307 between a <MOTION> tag and a </MOTION> tag.
This code 315 is "15FR_03". EVJ rendering/authoring functionality
312 parses through the text and identifies this code (step 402 in
FIG. 13). Code 315 is a code for the first element (element 15FR_03)
in the first sequence set (sequence set SET_01) illustrated in FIG.
5.
[0102] EVJ rendering/authoring functionality 312 contains information
on how to translate each element code that might appear in a scenario
file into a corresponding sequence of content manipulation instructions.
Content manipulation instructions may include instructions to blend
images from image files, zoom an image from an image file, pan an
image from an image file, tilt an image from an image file, flip
an image from an image file, rotate an image from an image file,
start playing audio from an audio file, stop playing audio from
an audio file, change the volume of audio being rendered, display
text on the screen, blend an audio snippet from an audio file with
music from another audio files, and so forth. Content may be still
image information, video information, audio snippet information,
music, and textual information.
[0103] In the illustrated example, EVJ file 300 does not contain
a list of content manipulation instructions for rendering the 15FR_03
element. This information is, however, known to the EVJ rendering/authoring
functionality 312. EVJ rendering/authoring functionality 312 translates
the code 315 into an appropriate sequence of content manipulation
instructions (step 403 of FIG. 13). The content manipulation instructions
are then carried out by the EVJ rendering/authoring functionality
312 in order and at the appropriate times so as to render the content
identified by the playlist file 308 in accordance with the element
(step 404 in FIG. 13). The EVJ rendering/authoring functionality
312 repeats this process, parsing through the text of scenario file
307 in order. Where rendering of an element calls for an additional
image, the additional image is the image contained in the next image
file listed in playlist file 308. The images that appear in the
slide show are therefore rendered in the order indicated in playlist
file 308. Similarly, when rendering of an element calls for an additional
amount of audio to be played, the additional audio is the audio
contained in the next audio file listed in playlist file 308.
[0104] In one embodiment, scenario file 307 includes a hierarchical
definition of a scenario, where one portion of the scenario file
points to another portion of the scenario file. A sequence set is
defined to be a sequence of elements, where the elements are taken
from a group of predetermined elements known to the EVJ rendering/authoring
functionalities of rendering devices. Multiple such sequence sets
are defined in the scenario file. A scenario is defined to be a
sequence of selected ones of the defined sequence sets. Once a sequence
set is defined, that definition can be referenced multiple times
in the definition of the scenario.
[0105] When an EVJ file is read by an EVJ rendering device and
is found not to include a scenario file, or where the EVJ rendering
device is unable to decipher the scenario file, then the EVJ rendering
device uses a default scenario to render the content found in the
EVJ file. The EVJ rendering device stores information on the default
scenario so that the default scenario need not be included in the
EVJ file.
[0106] In addition to the tags illustrated in FIG. 11, EVJ container
file 300 includes tags that identify a creator name field, a created
date field, a modified date field, and a size field. EVJ container
file 300 also contains a field that contains a code, where the code
indicates baseline hardware capabilities necessary to properly render
the file. The code may, for example, indicate that any images contained
in the file are of no more than a certain resolution or that the
images are in a certain format (for example, 4:2:2).
[0107] In one embodiment, EVJ rendering/authoring functionality
of a rending device is usable to view a slide show, stop the slide
show at a desired point (for example, using remote control device
314), identify a content being rendered at the time in the slide
show when the slide show was stopped, select the file name that
contains the identified content, and extract the content file. A
copy of the content file may, for example, then be stored on the
rendering device as a separate file from the EVJ file. Using this
mechanism, a person viewing a slide show can extract a file containing
a desired still image seen in the slide show. The file, once extracted
and copied, can be transferred from device to device as any other
file.
[0108] In one embodiment, a user can use the rendering device to
view the text in the EVJ file, to edit the text, and to store the
modified EVJ file. The order of images appearing when the EVJ file
is rendered can be changed by viewing the playlist 308, and changing
the order of the content file names appearing in the playlist, and
then saving the modified version of the EVJ file. Content file names
can be removed from a playlist in this fashion, and content file
names can be added in this fashion. Where the file name of a content
file is added to a playlist, the identified content file can also
be added to the EVJ file using the rendering device. All of these
editing functions can be controlled using the remote control device
314. Using remote control device 314, an EVJ file can be rendered,
altered, rendered again, altered again, and so forth such that the
ultimate EVJ file is customized in a manner desired by the user.
An ordinary text editor program can be used to edit a text file
within an EVJ file. An EVJ file can be copied, stored, and transferred
from computer to computer as other types of files are commonly copied,
stored and transferred. An EVJ file for a slide show can, for example,
be generated by a first person using a first rendering device, the
resulting EVJ file can then be emailed from the first person to
a second person, and then the second person can use a second rendering
device to read the EVJ file and render content contained in the
EVJ file so as to replay the slide show.
[0109] A personal desktop computer 316 is shown in FIG. 12 to illustrate
the point that an ordinary personal computer can be a rendering
device. The EVJ rendering/authoring functionality can be a software
implementation such that no special hardware other than standard
personal computer hardware is required to implement the rendering
device. Although the mode of transferring the EVJ file from one
rendering device to another rendering device in the example of FIG.
12 is a flash memory card, this mode is presented only as an example.
Any mechanism for communicating files from one electronic device
to another can be employed.
[0110] A rendering device may have a capability of storing the
slide show video stream on an optical disc such as a standard DVD
optical disc. The slide show can then be viewed by simply playing
the DVD (for example, using a DVD player and a television).
[0111] The EVJ rendering/authoring functionality 312 in television
311 may be implemented using the very same image processing integrated
circuit 103 employed in digital camera 3. By using the same integrated
circuit in a television rendering device that is already in use
in digital cameras, non-recurring engineering costs associated with
building the EVJ functionality into the television are reduced.
Where increased volumes of integrated circuit 103 are manufactured
due to the use of the same integrated circuit in televisions and
cameras, the unit cost of producing integrated circuit 103 can be
reduced when compared to the cost of producing integrated circuit
103 just for use in digital cameras.
[0112] Although the present invention has been described in connection
with certain specific embodiments for instructional purposes, the
present invention is not limited thereto. Accordingly, various modifications,
adaptations, and combinations of various features of the described
embodiments can be practiced without departing from the scope of
the invention as set forth in the claims.
|