How about reading an e-book with some music playing in the background, music that matches the mood of the portion of the e-book being read? Many feel this can lead to a more wholesome reading experience, perhaps more so for those who are both a bookworm and a music buff at the same time. Now researchers at the International Institute of Information Technology Hyderabad too have come up with just such an idea and have also devised an algorithm that they claim will sync perfectly with what one might be reading at the time, the publication, TheHinduBusinessLine reported.
The three-member team working on the project comprises of faculties Makarand Tapaswi and Vinoo Alluri along with Jaidev Shriram who is a BTech student. They are demoing their algorithm with a Harry Potter novel with music taken from the corresponding movie. They said they are right now focussing on e-books which have also been depicted on the silver screen. This is going to be easier as they have the right music ready that they can juxtapose on specific portions of the e-book.
“Harry Potter series was an obvious choice for us as its movie adaptations are big hits,” said Jaidev Shriram.
The researchers said their first task is to segment the e-book into portions depending on the mood being reflected in the particular portion. The soundtrack from the movie is also taken and adapted to play along with the e-book portion. However, care is being taken to prevent running into copyright issues.
“We categorized the music based on how homogenous it sounded in terms of emotion,” said Shriram.
However, one issue that the researchers said they have to deal with is that they don’t have music that can cover the entire length of the book or most parts of it. With the movie soundtrack filling in just parts of the e-book, they went for what they described as an ’emotion-based retrieval system’ to fill in the gaps.
“If a para evokes feelings of anger, gloom, or fear, then the algorithm would automatically choose a bit that reflects the mood,” Alluri said.
That way, the application should play the right music as you go along with the e-book. However, all of it is still in the early stages but it’s one concept that seems to have a lot of potentials. This can also be considered as sort of a cross between an audiobook and an e-book as there isn’t a voice narrating the book. Rather, it’s you who is reading the book but there is background music playing around that can be more fulfilling than while reading off it. That all of it sounds very interesting and does not need explaining.
Meanwhile, the research patent for which is pending has won the Brave New Idea Award at the 23rd International Society for Music Information Retrieval (ISMIR 2022) that was held in Bengaluru early this month.
With a keen interest in tech, I make it a point to keep myself updated on the latest developments in technology and gadgets. That includes smartphones or tablet devices but stretches to even AI and self-driven automobiles, the latter being my latest fad. Besides writing, I like watching videos, reading, listening to music, or experimenting with different recipes. The motion picture is another aspect that interests me a lot, and I'll likely make a film sometime in the future.
