Build Your Own Audio/Video Analytics App With HPE Haven OnDemand – Part 2

Build Your Own Audio/Video Analytics App With HPE Haven OnDemand – Part 2
In the conclusion to this two part tutorial, learn how to leverage HPE Haven OnDemand's Machine Learning APIs to build an audio/video analytics app with minimal time and effort.
By Phong Vu , HPE Haven OnDemand Developer Evangelist.
Editor's note: See part 1 of this tutorial , which was posted yesterday on KDnuggets.
Playback media content with Rich Media Analytics
List and search media content
Now it’s time to explore the power of Haven OnDemand text indexing which enables us to search for words or phrases from our media gallery.
To list all media content, we just call the Query Text Index API with the search argument defined as an asterisk “*”.
var arg = (searchVideos.Text.Length == 0) ? "*" : searchVideos.Text; var Params = new Dictionary
() { {"indexes", "speechindex"}, {"text", arg}, {"print_fields", "medianame,mediatype,filename"}, {"absolute_max_results", 100} }; hodApp = HODApps.QUERY_TEXT_INDEX; hodClient.PostRequest(ref Params, hodApp, HODClient.REQ_MODE.SYNC);
We will let the users enter search arguments into a text input field named “searchVideos”. Just for listing the media titles, we don’t need to read everything from the text index but just read the data we are interested in. We will use the “print_fields” to specify what piece of data we want to retrieve from the text index and in this case, they are the media name, media type and media file name.
We can also let the user choose to search only for audio or only video content by setting the “field_text” filter as shown in the code below. This is doable because we defined the mediatype field as a parametric field when we created our “speechindex” text index. If you want to provide a function to search for content from only certain languages, you can predefine the “language” field as a parametric field when you create the index and use similar syntax below for language.
if (media == "video") Params.Add("field_text", "MATCH{video/mp4}:mediatype"); else if (media == "audio") Params.Add("field_text", "MATCH{audio/mpeg,audio/mp4}:mediatype");
The Query Text Index API also includes optional parameters for autocomplete and spellcheck of user input search terms, which can further help improve the search experience for users.
Play media content with synchronized transcript
There are many different techniques in different platforms to synchronize the transcript while playing back the media content. In this demo, we will examine how the feature is implemented in Windows platform.
When a user selects a media content from the media list, we will call the Get Content API to retrieve the text and metadata of that media file from the index.
hodApp = HODApps.GET_CONTENT; var Params = new Dictionary
(); Params.Add("index_reference", item.reference); Params.Add("indexes", item.index); Params.Add("print_fields", "offset,text,content,concepts,occurrences,language"); hodClient.GetRequest(ref Params, hodApp, HODClient.REQ_MODE.SYNC);
When we called the Query Text Index, the API automatically returns matched item with mandatory fields such as the reference id and the index name where the item is indexed. Now, we use the reference id “index_reference” and the index name “indexes” to identify the item we want to get the content from.

 

Share it:
Share it:

[Social9_Share class=”s9-widget-wrapper”]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You Might Be Interested In

Artificial intelligence is healthcare’s next big thing

1 Feb, 2017

As the EMR “space race” peaks, clinical and health leaders are coming to understand that digitising data does not, on …

Read more

Big Data Is Making Medicine Personal

23 Mar, 2017

The use of big data in medicine is nothing new, we have seen it becoming more and more important across …

Read more

Maximizing Efficiency: The Key Role of Data Management Services

23 Feb, 2024

Unleash the power of data with top-notch data management services. Boost efficiency, make informed decisions, and drive business growth. Discover …

Read more

Do You Want to Share Your Story?

Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.

Get the 3 STEPS

To Drive Analytics Adoption
And manage change

3-steps-to-drive-analytics-adoption

Get Access to Event Discounts

Switch your 7wData account from Subscriber to Event Discount Member by clicking the button below and get access to event discounts. Learn & Grow together with us in a more profitable way!

Get Access to Event Discounts

Create a 7wData account and get access to event discounts. Learn & Grow together with us in a more profitable way!

Don't miss Out!

Stay in touch and receive in depth articles, guides, news & commentary of all things data.