Build Your Own Audio/Video Analytics App With HPE Haven OnDemand – Part 2
- by 7wData
Build Your Own Audio/Video Analytics App With HPE Haven OnDemand – Part 2
In the conclusion to this two part tutorial, learn how to leverage HPE Haven OnDemand's Machine Learning APIs to build an audio/video analytics app with minimal time and effort.
By Phong Vu , HPE Haven OnDemand Developer Evangelist.
Editor's note: See part 1 of this tutorial , which was posted yesterday on KDnuggets.
Playback media content with Rich Media Analytics
List and search media content
Now it’s time to explore the power of Haven OnDemand text indexing which enables us to search for words or phrases from our media gallery.
To list all media content, we just call the Query Text Index API with the search argument defined as an asterisk “*”.
var arg = (searchVideos.Text.Length == 0) ? "*" : searchVideos.Text; var Params = new Dictionary
() { {"indexes", "speechindex"}, {"text", arg}, {"print_fields", "medianame,mediatype,filename"}, {"absolute_max_results", 100} }; hodApp = HODApps.QUERY_TEXT_INDEX; hodClient.PostRequest(ref Params, hodApp, HODClient.REQ_MODE.SYNC);
We will let the users enter search arguments into a text input field named “searchVideos”. Just for listing the media titles, we don’t need to read everything from the text index but just read the data we are interested in. We will use the “print_fields” to specify what piece of data we want to retrieve from the text index and in this case, they are the media name, media type and media file name.
We can also let the user choose to search only for audio or only video content by setting the “field_text” filter as shown in the code below. This is doable because we defined the mediatype field as a parametric field when we created our “speechindex” text index. If you want to provide a function to search for content from only certain languages, you can predefine the “language” field as a parametric field when you create the index and use similar syntax below for language.
if (media == "video") Params.Add("field_text", "MATCH{video/mp4}:mediatype"); else if (media == "audio") Params.Add("field_text", "MATCH{audio/mpeg,audio/mp4}:mediatype");
The Query Text Index API also includes optional parameters for autocomplete and spellcheck of user input search terms, which can further help improve the search experience for users.
Play media content with synchronized transcript
There are many different techniques in different platforms to synchronize the transcript while playing back the media content. In this demo, we will examine how the feature is implemented in Windows platform.
When a user selects a media content from the media list, we will call the Get Content API to retrieve the text and metadata of that media file from the index.
hodApp = HODApps.GET_CONTENT; var Params = new Dictionary
(); Params.Add("index_reference", item.reference); Params.Add("indexes", item.index); Params.Add("print_fields", "offset,text,content,concepts,occurrences,language"); hodClient.GetRequest(ref Params, hodApp, HODClient.REQ_MODE.SYNC);
When we called the Query Text Index, the API automatically returns matched item with mandatory fields such as the reference id and the index name where the item is indexed. Now, we use the reference id “index_reference” and the index name “indexes” to identify the item we want to get the content from.
[Social9_Share class=”s9-widget-wrapper”]
Upcoming Events
Strategies for simplifying complex Salesforce data migrations – Free Webinar
27 March 2024
5 PM CET – 6 PM CET
Read MoreYou Might Be Interested In
Artificial intelligence is healthcare’s next big thing
1 Feb, 2017As the EMR “space race” peaks, clinical and health leaders are coming to understand that digitising data does not, on …
Big Data Is Making Medicine Personal
23 Mar, 2017The use of big data in medicine is nothing new, we have seen it becoming more and more important across …
Maximizing Efficiency: The Key Role of Data Management Services
23 Feb, 2024Unleash the power of data with top-notch data management services. Boost efficiency, make informed decisions, and drive business growth. Discover …
Recent Jobs
Do You Want to Share Your Story?
Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.