Abstract Submission: In the water industry, subject matter experts (SMEs) often manually review numerous documents and websites to find image-based information, such as survival curves for pipe deterioration and software manual screenshots for model setups. This process is time-consuming, inconsistent, and inefficient. This paper introduces a multimodal agentic Retrieval-Augmented Generation (RAG) tool to automate this task. RAG combines information retrieval systems with generative models to process and integrate multiple data types, including text, images, and tables.
To achieve our goal, we designed a unique nested agentic multimodal RAG with three main LLM (large language model) agents. The first agent, although optional, conducts research if no library of PDFs and websites is provided. For instance, it can locate top articles for a survival curve of cast iron pipes. The second agent, a Multimodal RAG specialist based on Llamaindex, extracts images, curves, tables, and text. Finally, the third agent digitizes the extracted curves into CSV format.
This framework has successfully produced accurate and reliable results for multiple test cases. In one example, the framework automatically analyzed PDFs and websites and provided survival curves and median break times for cast iron pipes. Another experiment focused on extracting both images and text from a software manual to answer questions about cost factors. This advanced automation framework leads to better decision-making, optimized performance, and increased efficiency in managing water infrastructure. Moreover, it significantly reduces the manual effort required by SMEs, enabling them to focus on higher-level analytical tasks.
Learning Objectives/Expected Outcome (Optional) : Learn about the concept and components of a multimodal agentic Retrieval-Augmented Generation (RAG) tool. Gain insights into the design and functionality of a nested agentic multimodal RAG with three main LLM agents. Knowledge of how a multimodal agentic RAG tool can automate the retrieval and integration of various data types. Recognize the potential impact of such automated systems on decision-making, performance optimization, and efficiency in managing water infrastructure. Awareness of real-world applications and case studies demonstrating the framework's effectiveness in producing accurate and reliable results.
