2024

Enhanced Media Experience with AI-Powered Commercial Detection and Replacement

Summary: Leveraging the capabilities of BeagleBoard’s powerful processing units, the project will focus on creating a real-time, efficient solution that enhances media consumption experiences by seamlessly integrating custom audio streams during commercial breaks.

• Develop a neural network model: Combine Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to analyze video and audio data, accurately identifying commercial segments within video streams.

• Implement a real-time pipeline: Create a real-time pipeline for BeagleBoard that utilizes the trained model to detect commercials in real-time and replace them with alternative content or obfuscate them, alongside replacing the audio with predefined streams.

• Optimize for BeagleBoard: Ensure the entire system is optimized for real-time performance on BeagleBoard hardware, taking into account its unique computational capabilities and constraints.

Contributor: Aryan Nanda

Mentors: Jason Kridner, Deepak Khatri, Kumar Abhishek

- GSoC Registry

Proposal

Low-latency I/O RISC-V CPU core in FPGA fabric

Summary: Implementation of PRU subsystem on BeagleV-Fire’s FPGA fabric, resulting in a real-time microcontroller system working alongside the main CPU, providing low-latency access to I/O.

Contributor: Atharva Kashalkar

Mentors: Cyril Jean, Jason Kridner, Vedant Paranjape, Kumar Abhishek

- GSoC Registry

Proposal

Differentiable Logic for Interactive Systems and Generative Music - Ian Clester

Summary: Developing an embedded machine learning system on BeagleBoard that leverages Differentiable Logic (DiffLogic) for real-time interactive music creation and environment sensing. The system will enable on-device learning, fine-tuning, and efficient processing for applications in new interfaces for musical expression.

Contributor: Ian Clester

Mentors: Jack Armitage, Chris Kiefer

- GSoC Registry

Proposal